[GitHub] carbondata pull request #1713: [CARBONDATA-1899] Optimize CarbonData concurr...

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2574/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2515/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1298/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2682/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    @jackylk Please review it


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2787/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1555/



---

[qiuchenjian-2]

Github user sraghunandan commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1744/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2974/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Please review it @jackylk


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1901/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3134/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    @xubo245 what is the memory requirements for this test? it fails for me even after I set the Xmx to 4GB.


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    @ravipesala It's ok in my computer with Xmx=4g. You can increase the value of Xmx or decrease the value of  totalNum.
   
    The  memory requirements is similar org.apache.carbondata.examples.CompareTest


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1713#discussion_r164676060
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/ConcurrencyTest.scala ---
    @@ -18,44 +18,60 @@
     package org.apache.carbondata.examples
     
     import java.io.File
    +import java.text.SimpleDateFormat
     import java.util
    +import java.util.Date
     import java.util.concurrent.{Callable, Executors, Future, TimeUnit}
     
     import scala.util.Random
     
     import org.apache.spark.sql.{DataFrame, Row, SaveMode, SparkSession}
     import org.apache.spark.sql.types._
     
    -import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.constants.{CarbonCommonConstants, CarbonVersionConstants}
     import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil}
     
     // scalastyle:off println
     object ConcurrencyTest {
    --- End diff --
   
    please make a benchmark package and move it there, rename the class to `ConcurrencyBenchmark`


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1713#discussion_r164718273
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/ConcurrencyTest.scala ---
    @@ -18,44 +18,60 @@
     package org.apache.carbondata.examples
     
     import java.io.File
    +import java.text.SimpleDateFormat
     import java.util
    +import java.util.Date
     import java.util.concurrent.{Callable, Executors, Future, TimeUnit}
     
     import scala.util.Random
     
     import org.apache.spark.sql.{DataFrame, Row, SaveMode, SparkSession}
     import org.apache.spark.sql.types._
     
    -import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.constants.{CarbonCommonConstants, CarbonVersionConstants}
     import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil}
     
     // scalastyle:off println
     object ConcurrencyTest {
    --- End diff --
   
    ok, done


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3268/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1713
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2032/



---