[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
GitHub user xubo245 opened a pull request:

    https://github.com/apache/carbondata/pull/2925

    [CARBONDATA-3102] Fix NoClassDefFoundError when use thriftServer and beeline

   
    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [ ] Any interfaces changed?
     No
     - [ ] Any backward compatibility impacted?
     No
     - [ ] Document update required?
    No
     - [ ] Testing done
        No
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    No


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xubo245/carbondata CARBONDATA-3102_fixBeelineThriftServerError

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2925.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2925
   
----
commit 86201f124c8857d70d50730fa33d93cd250d6d59
Author: xubo245 <xubo29@...>
Date:   2018-11-15T12:36:12Z

    [CARBONDATA-3102] Fix NoClassDefFoundError when use thriftServer and beeline

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1423/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9680/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1632/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    @jackylk @KanakaKumar @kunal642 Please review it.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1429/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1638/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9686/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    @jackylk @KanakaKumar @kunal642 Please review it.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    Please add description


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2925: [CARBONDATA-3102] Fix NoClassDefFoundError when use ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/2925
 
    @xubo245 I what scenario are you facing this issue? I am not facing this issue.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234191527
 
    --- Diff: integration/spark2/pom.xml ---
    @@ -134,6 +134,11 @@
             </exclusion>
           </exclusions>
         </dependency>
    +    <dependency>
    --- End diff --
   
    Why is this dependency needed in spark2?


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234191597
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala ---
    @@ -48,8 +48,13 @@ object CarbonThriftServer {
           System.exit(0)
         }
     
    +    val master = Option(System.getProperty("spark.master"))
    --- End diff --
   
    Was this the reason for NoClassDefFoundError?


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234541398
 
    --- Diff: integration/spark2/pom.xml ---
    @@ -134,6 +134,11 @@
             </exclusion>
           </exclusions>
         </dependency>
    +    <dependency>
    --- End diff --
   
    Because when I use org.apache.carbondata.spark.thriftserver.CarbonThriftServe and use beeline to connect, it throw NoClassDefFoundError。NoClassDefFoundError: org/apache/http/impl/conn/PoolingClientConnectionManager。  PoolingClientConnectionManager is belong to httpclient, so I add:
   
        <dependency>
          <groupId>org.apache.httpcomponents</groupId>
          <artifactId>httpclient</artifactId>
          <version>${httpclient.version}</version>
        </dependency>


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234541941
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala ---
    @@ -48,8 +48,13 @@ object CarbonThriftServer {
           System.exit(0)
         }
     
    +    val master = Option(System.getProperty("spark.master"))
    --- End diff --
   
    No, it's support CarbonThriftServer run in local/cluster, which convenient for users. Before if user want to run CarbonThriftServer in local, user need add master("local")


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234962085
 
    --- Diff: integration/spark2/pom.xml ---
    @@ -134,6 +134,11 @@
             </exclusion>
           </exclusions>
         </dependency>
    +    <dependency>
    --- End diff --
   
    But nobody else is facing this issue. And also none of our environments are getting this exception. Maybe u need to add httpclient jars to you cluster seperately?


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234962295
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala ---
    @@ -48,8 +48,13 @@ object CarbonThriftServer {
           System.exit(0)
         }
     
    +    val master = Option(System.getProperty("spark.master"))
    --- End diff --
   
    ok


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r234982825
 
    --- Diff: integration/spark2/pom.xml ---
    @@ -134,6 +134,11 @@
             </exclusion>
           </exclusions>
         </dependency>
    +    <dependency>
    --- End diff --
   
    I meet more than two times. This issue also happen in spark on k8s before. You can try in original cluster.
    It's not convenient for user to  add httpclient jars to you cluster seperately.  What's effect if we add this dependency?


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user akashrn5 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r235015512
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala ---
    @@ -157,8 +157,8 @@ object S3Example {
       }
     
       def getSparkMaster(args: Array[String]): String = {
    -      if (args.length == 5) args(4)
    -      else if (args(3).contains("spark:") || args(3).contains("mesos:")) args(3)
    -      else "local"
    -    }
    --- End diff --
   
    revert the unnecessary changes


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2925: [CARBONDATA-3102] Fix NoClassDefFoundError wh...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2925#discussion_r235027703
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala ---
    @@ -157,8 +157,8 @@ object S3Example {
       }
     
       def getSparkMaster(args: Array[String]): String = {
    -      if (args.length == 5) args(4)
    -      else if (args(3).contains("spark:") || args(3).contains("mesos:")) args(3)
    -      else "local"
    -    }
    --- End diff --
   
    the format is incorrect. we should optimize it.
    ```def getS3EndPoint(args: Array[String]): String = {   def getS3EndPoint(args: Array[String]): String = {
         if (args.length >= 4 && args(3).contains(".com")) args(3)     if (args.length >= 4 && args(3).contains(".com")) args(3)
         else ""     else ""
       }```


---
12