[GitHub] [carbondata] VenuReddy2103 commented on a change in pull request #3819: [CARBONDATA-3855]support carbon SDK to load data from different files

[GitBox]

VenuReddy2103 commented on a change in pull request #3819:
URL: https://github.com/apache/carbondata/pull/3819#discussion_r478333555



##########
File path: sdk/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java
##########
@@ -660,13 +1113,42 @@ public CarbonWriter build() throws IOException, InvalidLoadOptionException {
       // removed from the load. LoadWithoutConverter flag is going to point to the Loader Builder
       // which will skip Conversion Step.
       loadModel.setLoadWithoutConverterStep(true);
-      return new AvroCarbonWriter(loadModel, hadoopConf, this.avroSchema);
+      AvroCarbonWriter avroCarbonWriter = new AvroCarbonWriter(loadModel,
+          hadoopConf, this.avroSchema);
+      if (!StringUtils.isEmpty(filePath)) {
+        avroCarbonWriter.setDataFiles(this.dataFiles);
+      }
+      return avroCarbonWriter;
     } else if (this.writerType == WRITER_TYPE.JSON) {
       loadModel.setJsonFileLoad(true);
-      return new JsonCarbonWriter(loadModel, hadoopConf);
+      JsonCarbonWriter jsonCarbonWriter = new JsonCarbonWriter(loadModel, hadoopConf);
+      if (!StringUtils.isEmpty(filePath)) {
+        jsonCarbonWriter.setDataFiles(this.dataFiles);
+      }
+      return jsonCarbonWriter;
+    } else if (this.writerType == WRITER_TYPE.PARQUET) {
+      loadModel.setLoadWithoutConverterStep(true);
+      AvroCarbonWriter avroCarbonWriter = new AvroCarbonWriter(loadModel,
+          hadoopConf, this.avroSchema);
+      ParquetCarbonWriter parquetCarbonWriter = new
+          ParquetCarbonWriter(avroCarbonWriter, hadoopConf);

Review comment:
       Instead of creating instance of `AvroCarbonWriter` and passing it to `ParquetCarbonWriter`, create `AvroCarbonWriter` instance internally within the constructor of `ParquetCarbonWriter` Suggest same for `ORCCarbonWriter` below




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]