[jira] [Created] (CARBONDATA-3389) Optimize the bundling of Hadoop libraries with CarbonData

[Akash R Nilugal (Jira)]

lamber-ken created CARBONDATA-3389:
--------------------------------------

             Summary: Optimize the bundling of Hadoop libraries with CarbonData
                 Key: CARBONDATA-3389
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3389
             Project: CarbonData
          Issue Type: Improvement
          Components: hadoop-integration
    Affects Versions: 1.5.3
            Reporter: lamber-ken
             Fix For: NONE


For now, CarbonData provides archives bunding with hadoop-2.7.2, user needs to build carbondata to fit their own hadoop env.
{code:java}
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.1.0-hadoop2.7.2.jar.sha512
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.2.1-hadoop2.7.2.jar.sha512
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar.asc
apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar.sha512
{code}
 

I think it's better to split carbondata and hadoop. use can manually download a pre-packaged Hadoop jar from the optional components, like bellow
{code:java}
CarbonData 1.6.0
CarbonData 1.6.0 for Scala 2.11
CarbonData 1.6.0 for Scala 2.12

Optional components
Pre-bundled Hadoop 2.4.1
Pre-bundled Hadoop 2.6.5
Pre-bundled Hadoop 2.7.5
Pre-bundled Hadoop 2.8.3{code}
 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)