[GitHub] carbondata pull request #2738: [CARBONDATA-2952] Provide c++ interface for S...

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220833721
 
    --- Diff: store/CSDK/CarbonReader.cpp ---
    @@ -0,0 +1,97 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +#include "CarbonReader.h"
    +#include <jni.h>
    +
    +jobject CarbonReader::builder(JNIEnv *env, char *path, char *tableName) {
    +
    +    jniEnv = env;
    +    jclass carbonReaderClass = env->FindClass("org/apache/carbondata/sdk/file/CarbonReader");
    +    jmethodID carbonReaderBuilderID = env->GetStaticMethodID(carbonReaderClass, "builder",
    +        "(Ljava/lang/String;Ljava/lang/String;)Lorg/apache/carbondata/sdk/file/CarbonReaderBuilder;");
    +    jstring jpath = env->NewStringUTF(path);
    +    jstring jtableName = env->NewStringUTF(tableName);
    +    jvalue args[2];
    +    args[0].l = jpath;
    +    args[1].l = jtableName;
    +    carbonReaderBuilderObject = env->CallStaticObjectMethodA(carbonReaderClass, carbonReaderBuilderID, args);
    +    return carbonReaderBuilderObject;
    +}
    +
    +jobject CarbonReader::projection(int argc, char *argv[]) {
    +    jclass carbonReaderBuilderClass = jniEnv->GetObjectClass(carbonReaderBuilderObject);
    +    jmethodID buildID = jniEnv->GetMethodID(carbonReaderBuilderClass, "projection",
    +        "([Ljava/lang/String;)Lorg/apache/carbondata/sdk/file/CarbonReaderBuilder;");
    +    jclass objectArrayClass = jniEnv->FindClass("Ljava/lang/String;");
    +    jobjectArray array = jniEnv->NewObjectArray(argc, objectArrayClass, NULL);
    +    for (int i = 0; i < argc; ++i) {
    +        jstring value = jniEnv->NewStringUTF(argv[i]);
    +        jniEnv->SetObjectArrayElement(array, i, value);
    +    }
    +
    +    jvalue args[1];
    +    args[0].l = array;
    +    carbonReaderBuilderObject = jniEnv->CallObjectMethodA(carbonReaderBuilderObject, buildID, args);
    +    return carbonReaderBuilderObject;
    +}
    +
    +jobject CarbonReader::withHadoopConf(int argc, char *argv[]) {
    --- End diff --
   
    OK, I added api:config(char *key, char *value)


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220833952
 
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java ---
    @@ -101,6 +101,20 @@ public CarbonReaderBuilder withHadoopConf(Configuration conf) {
         return this;
       }
     
    +
    +  public CarbonReaderBuilder withHadoopConf(String[] args) {
    --- End diff --
   
     I added api:config(char *key, char *value), only in c layer


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/602/



---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220851174
 
    --- Diff: store/CSDK/CarbonReader.h ---
    @@ -0,0 +1,105 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +#ifndef TM_SCALA_CARBONREADER_H
    +#define TM_SCALA_CARBONREADER_H
    +
    +
    +#include <jni.h>
    +#include <string>
    --- End diff --
   
    remove


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/794/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8861/



---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220907838
 
    --- Diff: store/sdk/pom.xml ---
    @@ -39,6 +39,16 @@
           <artifactId>hadoop-aws</artifactId>
           <version>${hadoop.version}</version>
         </dependency>
    +    <dependency>
    +      <groupId>org.apache.httpcomponents</groupId>
    +      <artifactId>httpclient</artifactId>
    +      <version>4.2</version>
    +    </dependency>
    +    <dependency>
    +      <groupId>org.apache.hadoop</groupId>
    +      <artifactId>hadoop-common</artifactId>
    --- End diff --
   
    removed  now


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220912011
 
    --- Diff: store/sdk/pom.xml ---
    @@ -39,6 +39,16 @@
           <artifactId>hadoop-aws</artifactId>
           <version>${hadoop.version}</version>
         </dependency>
    +    <dependency>
    +      <groupId>org.apache.httpcomponents</groupId>
    +      <artifactId>httpclient</artifactId>
    --- End diff --
   
    ok, done.support hadoop 2.7.2 and 2.8.3


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2738#discussion_r220912378
 
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java ---
    @@ -101,6 +101,20 @@ public CarbonReaderBuilder withHadoopConf(Configuration conf) {
         return this;
       }
     
    +
    +  public CarbonReaderBuilder withHadoopConf(String[] args) {
    +    Configuration configuration = new Configuration();
    --- End diff --
   
    I add config(String key, String value)  for it.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/608/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/610/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/802/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8871/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/617/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8878/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/809/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/618/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8879/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2738
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/810/



---