Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244069761 --- Diff: store/CSDK/test/main_ft.cpp --- @@ -0,0 +1,1172 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include <iostream> --- End diff -- Can you split different function test to different files? maybe can refer: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/framework, https://github.com/tensorflow/tensorflow/blob/81fa63d5dd34b9a8046f79c16731efa4dee9dd63/tensorflow/core/BUILD --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244069960 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK --- End diff -- Can you add some doc for CLion? CLion also support compile/build, run, debug. --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244070055 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK --- End diff -- Please view the file. the md format is different in GitHub, please change ##Compile to ## Compile. --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244071727 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` + +Test Cases are written in [main.cpp](https://github.com/apache/carbondata/blob/master/store/CSDK/test/main.cpp) with GoogleTest C++ Framework. +if GoogleTest LIBRARY is not added then compilation of example code will fail. Please follow below steps to solve the same --- End diff -- In CLionï¼main.cpp don't fail when GoogleTest LIBRARY is not added --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1965/ --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244116001 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` --- End diff -- How to run it by command? --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244116085 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` + +Test Cases are written in [main.cpp](https://github.com/apache/carbondata/blob/master/store/CSDK/test/main.cpp) with GoogleTest C++ Framework. +if GoogleTest LIBRARY is not added then compilation of example code will fail. Please follow below steps to solve the same +1. Remove test/main.cpp from SOURCE_FILES of CMakeLists.txt and compile/build again. +2. Follow below Steps to configure GoogleTest Framework + * Download googleTest release (CI is complied with 1.8) https://github.com/google/googletest/releases + * Extract to folder like /opt/googletest/googletest-release-1.8.1/ and create build folder inside this like /opt/googletest/googletest-release-1.8.1/googletest/build) + * build googleTest , Goto build path and run `cmake ../` . this will generate google lib in build folder. + * [Optional] it is optional if GTest lib can be found in system lib while running `cmake` command for build. + <br>copy google lib and include folder to respective path + * `cp /opt/googletest/googletest-release-1.8.1/googletest/build/*.a /usr/local/lib/` + * `cp -r /opt/googletest/googletest-release-1.8.1/googletest/include/gtest /usr/local/include/` --- End diff -- I already make googletest in my local, how to run googletest? --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10217/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2238/ --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244507492 --- Diff: store/CSDK/test/main.cpp --- @@ -665,6 +699,208 @@ bool readFromS3(JNIEnv *env, char *path, char *argv[]) { printResult(env, reader); } +TEST(CSDKTest,tryCatchException) { + bool gotExp=tryCatchException(env); + EXPECT_TRUE(gotExp); +} + + +TEST(CSDKTest,tryCarbonRowException) { + char *smallFilePath = "../../../../resources/carbondata"; + try { + bool result = tryCarbonRowException(env, smallFilePath);; + EXPECT_TRUE(result) << "Expected Exception as No Index File" << result; + } catch (runtime_error e) { + EXPECT_TRUE(true); + } +} + +TEST(CSDKTest,testCarbonProperties) { + try { + bool result = testCarbonProperties(env); + EXPECT_TRUE(result) << "Carbon set properties not working "; + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected while setting carbon properties "; + } +} + +TEST(CSDKTest,testWriteData) { + try { + bool result =testWriteData(env, "./data", my_argc, my_argv); + result = result && testWriteData(env, "./data", my_argc, my_argv); + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected while data loading"; + } +} + +TEST(CSDKTest,readFromLocalWithoutProjection) { + try { + char *smallFilePath = "./data_withoutpro"; + bool result =testWriteData(env, smallFilePath, my_argc, my_argv); + if(result){ + bool proj_result = readFromLocalWithoutProjection(env, smallFilePath); + EXPECT_TRUE(proj_result) << "Without Projection is failed"; + } else { + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected ,During without projection selection"; + } +} + +TEST(CSDKTest,readFromLocalWithProjection) { + try { + char *smallFilePath = "./data_pro"; + bool result =testWriteData(env, smallFilePath, my_argc, my_argv); + if(result){ + bool proj_result = readFromLocalWithProjection(env, smallFilePath); + EXPECT_TRUE(proj_result) << "With Projection is failed"; + } else { + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected ,During With projection selection"; + } +} + + +TEST(CSDKTest,readSchemaWithoutValidation) { + try { + char *path = "./data_readPath"; + bool result =testWriteData(env, path, my_argc, my_argv); + if(result){ + bool schema_result = readSchema(env, path, false); + EXPECT_TRUE(schema_result) << "Not Able to read readSchema from given path"; + } else { + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected ,During Read Schema"; + } +} + + +TEST(CSDKTest,readSchemaWithValidation) { + try { + char *path = "./data_readPathWithValidation"; + bool result =testWriteData(env, path, my_argc, my_argv); + if(result){ + bool schema_result = readSchema(env, path, true); + EXPECT_TRUE(schema_result) << "Not Able to read readSchema Or validate from given path"; + } else { + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected ,During Read Schema"; + } +} + +TEST(CSDKTest,testReadNextRowWthtVector) { + try { + int printNum = 32000; + char *path = "./data_forVector"; + bool result =testWriteData(env, path, my_argc, my_argv); + if(result){ + bool readresultWithVector= testReadNextRow(env, path, printNum, my_argv, 0, true); + EXPECT_TRUE(readresultWithVector) << "Vector reading is failed"; + } else { + EXPECT_TRUE(result) << "Either Data Loading Or Carbon Reader is failed"; + } + } catch (runtime_error e) { + EXPECT_TRUE(false) << " Exception is not expected ,During Vector read"; + } +} + + +TEST(CSDKTest,testReadNextRowWthoutVector) { + try { + int printNum = 32000; + char *path = "./data_forCarbonReader"; + bool result =testWriteData(env, path, my_argc, my_argv); + if(result){ + bool readresultWithVectorfalse=testReadNextRow(env, path, printNum, my_argv, 0, false); --- End diff -- added main_ft class for all functional test. --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244507596 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` + +Test Cases are written in [main.cpp](https://github.com/apache/carbondata/blob/master/store/CSDK/test/main.cpp) with GoogleTest C++ Framework. +if GoogleTest LIBRARY is not added then compilation of example code will fail. Please follow below steps to solve the same --- End diff -- Done for required words. --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244507697 --- Diff: docs/csdk-guide.md --- @@ -40,6 +66,7 @@ release the memory and destroy JVM. C++ SDK support read batch row. User can set batch by using withBatch(int batch) before build, and read batch by using readNextBatchRow(). + --- End diff -- removed extra line --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244508238 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK --- End diff -- Clion is not used for deployment of CSDK . it is IDE which is not mandatory to use CSDK . For developer, Clion, Eclipse(CDT)..etc options are available. and developer can refer respective IDE help about how to import cmake project.(added in carbon sdk guide about this) --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244508413 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK --- End diff -- Fixed. added same like C++ SDK Reader --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244508891 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` + +Test Cases are written in [main.cpp](https://github.com/apache/carbondata/blob/master/store/CSDK/test/main.cpp) with GoogleTest C++ Framework. +if GoogleTest LIBRARY is not added then compilation of example code will fail. Please follow below steps to solve the same --- End diff -- Test class (now main_ft.cpp ) has #include "gtest/gtest.h" and TEST annotation which required GoogleTest. Now i have handled in CmakeList.txt in which if googleTest is not found then test class will not be added in source file in this way build will not fail. --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244509021 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` --- End diff -- cmake,make are to be executed in terminal. --- |
In reply to this post by qiuchenjian-2
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2991#discussion_r244509375 --- Diff: docs/csdk-guide.md --- @@ -29,6 +29,32 @@ code and without CarbonSession. In the carbon jars package, there exist a carbondata-sdk.jar, including SDK reader for C++ SDK. + +##Compile/Build CSDK +CSDK supports cmake based compilation and has dependency list in CMakeLists.txt. +<br> Prerequisites +<br>GCC >=4.8.5 +<br>Cmake >3.13 +<br>Make >=4.1 + +Steps +1. Go to CSDK folder(/opt/.../CSDK/) +2. Create build folder . (/opt/.../CSDK/build) +3. Run Command from build folder `cmake ../` +4. `make` + +Test Cases are written in [main.cpp](https://github.com/apache/carbondata/blob/master/store/CSDK/test/main.cpp) with GoogleTest C++ Framework. +if GoogleTest LIBRARY is not added then compilation of example code will fail. Please follow below steps to solve the same +1. Remove test/main.cpp from SOURCE_FILES of CMakeLists.txt and compile/build again. +2. Follow below Steps to configure GoogleTest Framework + * Download googleTest release (CI is complied with 1.8) https://github.com/google/googletest/releases + * Extract to folder like /opt/googletest/googletest-release-1.8.1/ and create build folder inside this like /opt/googletest/googletest-release-1.8.1/googletest/build) + * build googleTest , Goto build path and run `cmake ../` . this will generate google lib in build folder. + * [Optional] it is optional if GTest lib can be found in system lib while running `cmake` command for build. + <br>copy google lib and include folder to respective path + * `cp /opt/googletest/googletest-release-1.8.1/googletest/build/*.a /usr/local/lib/` + * `cp -r /opt/googletest/googletest-release-1.8.1/googletest/include/gtest /usr/local/include/` --- End diff -- In IDE(Clion) you can run just like normal test case class (you can select individual testcase also). and without IDE you can run executable file ( which is defined in CmakeLists.txt). about how to build executable file ,you can refer csdk guide (Compile/build CSDK section). --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2086/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10340/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2991 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2291/ --- |
Free forum by Nabble | Edit this page |