[jira] [Created] (CARBONDATA-909) Single pass option in dataframe writer

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (CARBONDATA-909) Single pass option in dataframe writer

Akash R Nilugal (Jira)
Sanoj MG created CARBONDATA-909:
-----------------------------------

             Summary: Single pass option in dataframe writer
                 Key: CARBONDATA-909
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-909
             Project: CarbonData
          Issue Type: Bug
          Components: spark-integration
    Affects Versions: 1.2.0-incubating, 1.1.1-incubating
         Environment: HDP 2.5 / Spark 1.6
            Reporter: Sanoj MG
            Assignee: Sanoj MG
            Priority: Minor
             Fix For: 1.2.0-incubating, 1.1.1-incubating


While creating a Carbondata table from dataframe, currently it is not possible to specify single pass load in Spark 1.6. An option is required to specify it as below :

df.write.format("carbondata")
.option("tableName", "test")
.option("compress","true")
.option("single_pass","true")
.mode(SaveMode.Overwrite)
.save()



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)