[GitHub] carbondata pull request #1643: [CARBONDATA-1883] Improvement in merge index ...

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1643: [CARBONDATA-1883] Improvement in merge index ...

qiuchenjian-2
GitHub user dhatchayani opened a pull request:

    https://github.com/apache/carbondata/pull/1643

    [CARBONDATA-1883] Improvement in merge index code

    (1) Improved merge index code
    (2) Added trigger point for merge index
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [ ] Testing done
           Manual Testing
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhatchayani/incubator-carbondata improve_mergeIndex

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1643.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1643
   
----
commit 85bea4b2c70c0ed5d58eea927cf5bd120e2ec6ec
Author: dhatchayani <[hidden email]>
Date:   2017-12-12T06:32:15Z

    [CARBONDATA-1883] Improvement in merge index code

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/656/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1886/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2231/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1643: [CARBONDATA-1883] Improvement in merge index ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1643#discussion_r156416528
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAlterTableCompactionCommand.scala ---
    @@ -199,22 +199,34 @@ case class CarbonAlterTableCompactionCommand(
                     carbonTable.getAbsoluteTableIdentifier).asScala,
                   carbonLoadModel.getTablePath,
                   carbonTable, true)
    -            lock.unlock()
    -            return
    +
    +            // trigger event for merge index
    +            val operationContext = new OperationContext
    +            val alterTableCompactionPreEvent: AlterTableCompactionPreEvent =
    +              AlterTableCompactionPreEvent(sqlContext.sparkSession,
    +                carbonTable,
    +                null,
    +                "")
    +            OperationListenerBus.getInstance
    +              .fireEvent(alterTableCompactionPreEvent, operationContext)
    +
    +          } else {
    +            CarbonDataRDDFactory.startCompactionThreads(
    +              sqlContext,
    +              carbonLoadModel,
    +              storeLocation,
    +              compactionModel,
    +              lock,
    +              operationContext
    +            )
               }
    -          CarbonDataRDDFactory.startCompactionThreads(
    -            sqlContext,
    -            carbonLoadModel,
    -            storeLocation,
    -            compactionModel,
    -            lock,
    -            operationContext
    -          )
             } catch {
               case e: Exception =>
                 LOGGER.error(s"Exception in start compaction thread. ${ e.getMessage }")
                 lock.unlock()
    --- End diff --
   
    Remove lock.unlock() from catch block


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    retest sdv please


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1910/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/681/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2242/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    retest sdv please


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2243/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1643: [CARBONDATA-1883] Improvement in merge index ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1643


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1643: [CARBONDATA-1883] Improvement in merge index code

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1643
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2246/



---