[GitHub] carbondata pull request #1604: Basic2

[qiuchenjian-2]

GitHub user jackylk opened a pull request:

    https://github.com/apache/carbondata/pull/1604

    Basic2

    1. Block 'external' syntax
    2. Block 'CTAS' syntax
   
     - [X] Any interfaces changed?
     No
     - [X] Any backward compatibility impacted?
     No
     - [X] Document update required?
    No
     - [X] Testing done
    Added
     - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    NA

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jackylk/incubator-carbondata basic2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1604.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1604
   
----
commit 4c1855b158a1f4e5840a3ed6ec8fd8c83c8bbb1f
Author: Jacky Li <[hidden email]>
Date:   2017-12-02T14:46:56Z

    add config

commit 0ecb111d933e33b91db5dcaeb9ea4980af3e29eb
Author: Jacky Li <[hidden email]>
Date:   2017-12-03T14:56:16Z

    change block

----


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1604#discussion_r154526332
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSession.scala ---
    @@ -58,7 +59,12 @@ class CarbonSession(@transient val sc: SparkContext,
        */
       @transient
      override lazy val sharedState: SharedState = {
    -    existingSharedState.getOrElse(new SharedState(sparkContext))
    +    if (existingSharedState.isDefined) {
    --- End diff --
   
    Better use `match {case}`


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1604#discussion_r154526536
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSession.scala ---
    @@ -58,7 +59,12 @@ class CarbonSession(@transient val sc: SparkContext,
        */
       @transient
      override lazy val sharedState: SharedState = {
    -    existingSharedState.getOrElse(new SharedState(sparkContext))
    +    if (existingSharedState.isDefined) {
    --- End diff --
   
    ok, fixed


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1655/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2043/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2044/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2045/



---

[qiuchenjian-2]

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1604#discussion_r154548820
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSession.scala ---
    @@ -170,8 +181,15 @@ object CarbonSession {
             }
             options.foreach { case (k, v) => session.sessionState.conf.setConfString(k, v) }
             SparkSession.setDefaultSession(session)
    -        CommonUtil.cleanInProgressSegments(
    -          carbonProperties.getProperty(CarbonCommonConstants.STORE_LOCATION), sparkContext)
    +        try {
    +          CommonUtil.cleanInProgressSegments(
    +            carbonProperties.getProperty(CarbonCommonConstants.STORE_LOCATION), sparkContext)
    +        } catch {
    +          case e: Throwable =>
    +            // catch all exceptions to avoid CarbonSession initialization failure
    +          LogServiceFactory.getLogService(this.getClass.getCanonicalName)
    +              .error(e, "Failed to clean in progress segments")
    --- End diff --
   
    indentation is incorrect


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2049/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1604
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1659/



---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1604


---