[GitHub] carbondata pull request #2221: [CARBONDATA-2392] Add close method for Carbon...

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184665460
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    --- End diff --
   
    move buildWriterForCSVInput to line 47


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184665508
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    +
    +        for (int i = 0; i < 10; i++) {
    +            writer.write(new String[]{"robot" + (i % 10), String.valueOf(i)});
    +        }
    +        writer.close();
    +
    +        // Read data
    +        CarbonReader reader = CarbonReader.builder(path, "_temp")
    +                .projection(new String[]{"name", "age"}).build();
    --- End diff --
   
    move build() to next line


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184665587
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    +
    +        for (int i = 0; i < 10; i++) {
    +            writer.write(new String[]{"robot" + (i % 10), String.valueOf(i)});
    +        }
    +        writer.close();
    +
    +        // Read data
    +        CarbonReader reader = CarbonReader.builder(path, "_temp")
    --- End diff --
   
    what is "_temp"?


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184665705
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    --- End diff --
   
    remove throws Exception


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184665776
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    --- End diff --
   
    Please describe this example, it is not testing


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4300/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5468/



---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184834326
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    --- End diff --
   
    ok,done


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184834335
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    +
    +        for (int i = 0; i < 10; i++) {
    +            writer.write(new String[]{"robot" + (i % 10), String.valueOf(i)});
    +        }
    +        writer.close();
    +
    +        // Read data
    +        CarbonReader reader = CarbonReader.builder(path, "_temp")
    +                .projection(new String[]{"name", "age"}).build();
    --- End diff --
   
    ok, done


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184834374
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    +        String path = "./testWriteFiles";
    +        FileUtils.deleteDirectory(new File(path));
    +
    +        Field[] fields = new Field[2];
    +        fields[0] = new Field("name", DataTypes.STRING);
    +        fields[1] = new Field("age", DataTypes.INT);
    +
    +        CarbonWriterBuilder builder = CarbonWriter.builder()
    +                .withSchema(new Schema(fields))
    +                .isTransactionalTable(true)
    +                .outputPath(path)
    +                .persistSchemaFile(true);
    +
    +        CarbonWriter writer = builder.buildWriterForCSVInput();
    +
    +        for (int i = 0; i < 10; i++) {
    +            writer.write(new String[]{"robot" + (i % 10), String.valueOf(i)});
    +        }
    +        writer.close();
    +
    +        // Read data
    +        CarbonReader reader = CarbonReader.builder(path, "_temp")
    --- End diff --
   
    temp table name


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184834524
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    + */
    +public class CarbonReaderExample {
    +    public static void main(String[] args) throws Exception {
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2221#discussion_r184834664
 
    --- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.examples.sdk;
    +
    +import java.io.File;
    +import java.io.FilenameFilter;
    +import java.util.List;
    +
    +import org.apache.commons.io.FileUtils;
    +
    +import org.apache.carbondata.core.metadata.datatype.DataTypes;
    +import org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
    +import org.apache.carbondata.sdk.file.*;
    +
    +/**
    + * Example for testing CarbonReader
    --- End diff --
   
    ok, done


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4321/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5487/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4590/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    retest this please


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4594/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    retest sdv please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2221
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4328/



---