[GitHub] carbondata pull request #2673: [WIP] Test Carbonstore

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2673: [WIP] Test Carbonstore

qiuchenjian-2
GitHub user xubo245 opened a pull request:

    https://github.com/apache/carbondata/pull/2673

    [WIP] Test Carbonstore

    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [ ] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xubo245/carbondata carbonstoreCS

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2673.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2673
   
----
commit e80126a208a173225c9899df8e4a9f177a07636d
Author: Jacky Li <jacky.likun@...>
Date:   2018-06-13T15:57:00Z

    [CARBONDATA-2609] Change RPC implementation to Hadoop RPC framework
   
    This closes #2372

commit 437556bb8d44380836956cc341718bb56dce5cdf
Author: QiangCai <qiangcai@...>
Date:   2018-07-03T12:21:18Z

    [CARBONDATA-2690][CarbonStore] implement RESTful API: create table, load data and select
   
    This PR adds:
    1.basic framework
    rewrite the carbon store's Master, Worker and Scheduler code in Java
   
    2.RESTful API
    support create a table by using file meta store
    support load data to a table in single work
    support select data with a filter
   
    This closes #2440

commit 01c623d67dc0384674b2e489a88d819f687d6ac6
Author: xuchuanyin <xuchuanyin@...>
Date:   2018-06-13T01:03:28Z

    [CARBONDATA-2613] Support csv based carbon table
   
    1. create csv based carbon table using
    CREATE TABLE fact_table (col1 bigint, col2 string, ..., col100 string)
    STORED BY 'CarbonData'
    TBLPROPERTIES(
      'foramt'='csv',
      'csv.delimiter'=',',
      'csv.header'='col1,col2,col100')
   
    2. Load data to this table using
    ALTER TABLE fact_table ADD SEGMENT LOCATION 'path/to/data1'
   
    This closes #2374

commit 464e948d55f43a71d82c5fb654f4e1ea66c1ef65
Author: Jacky Li <jacky.likun@...>
Date:   2018-07-09T04:23:49Z

    [CARBONDATA-2705][CarbonStore] CarbonStore Java API and Implementation
   
    Support two implementations:
    1.LocalCarbonStore for usage in local mode
    2.DistributedCarbonStore leveraging multiple server (Master and Workers) via RPC
   
    This closes #2473

commit 4cd7f5881b5595cf2b321dcb295a270162e53623
Author: Jacky Li <jacky.likun@...>
Date:   2018-07-10T13:20:45Z

    [CARBONDATA-2688][CarbonStore] Support SQL in REST API
   
    Support SQL interface in Horizon service.
    Support REST client for SQL
   
    This closes #2481

commit 72bdef53c6582e2b8127430675c1324c73ddc825
Author: Jacky Li <jacky.likun@...>
Date:   2018-07-18T02:14:43Z

    [REBASE] Rebasing with master branch and Fixing rebase conflict

commit a2d5b098d0764c61044039cc519601995347ae6d
Author: Ajith <ajith2489@...>
Date:   2018-07-12T03:47:22Z

    [CARBONDATA-2736][CARBONSTORE] Kafka integration with Carbon StreamSQL
   
    Modification in this PR:
    1.Pass source table properties to streamReader.load()
    2.Do not pass schema when sparkSession.readStream
    3.Remove querySchema validation against sink as dataFrame made from kafka source will not have schema ( its written in value column of schema )
    4.Extract the dataframe from kafka source which contain actual data schema @ writeStream
   
    This closes #2495

commit 8a63df595a10e2dbf03f08b108859f339ea4fe7a
Author: Ajith <ajith2489@...>
Date:   2018-07-18T11:18:54Z

    [CARBONDATA-2752][CARBONSTORE] Carbon provide Zeppelin support
   
    This closes #2522

commit 65249c0aaa5ff50452f5df24477d0f26e06a70b4
Author: QiangCai <qiangcai@...>
Date:   2018-07-19T06:50:38Z

    [CARBONDATA-2767][CarbonStore] Fix task locality issue
   
    If the Spark cluster and the Hadoop cluster are two different machine cluster, the Spark tasks will run in RACK_LOCAL mode.
   
    This closes #2528

commit f75a5c861ce00df9e00c4dbe50433d8fcf07ad0f
Author: QiangCai <qiangcai@...>
Date:   2018-07-24T03:18:59Z

    [CARBONDATA-2776][CarbonStore] Support ingesting data from Kafka service
   
    This closes #2544

commit 7443cd4da0d0d4d457456490a80995cdea7f7019
Author: Jacky Li <jacky.likun@...>
Date:   2018-07-31T18:16:26Z

    [CARBONDATA-2825][CARBONDATA-2828] CarbonStore and InternalCarbonStore API
    This closes #2589

commit 87706e04887383b58eca42a4477abf14969a63d8
Author: Ajith <ajith2489@...>
Date:   2018-08-12T11:24:27Z

    [CARBONDATA-2826] support select using distributed carbon store
   
    Provides select support with select columns pruning and filter pushdown using new RDD for distributed carbon store
   
    This closes 2631

commit 19e55a3a21cc3e4ab091863ee320917e05764a98
Author: Jacky Li <jacky.likun@...>
Date:   2018-08-17T08:03:10Z

    [HOTFIX] fix findbugs and pom dependency

commit 1233c61de3ed5ad577b962766f9e94ab9d62e2a1
Author: xubo245 <xubo29@...>
Date:   2018-08-23T02:27:56Z

    Test carbon performance
    1.suppport explain in zeppelin
    2.PageViewsProducer
    3.change the dis dependency
    4.add time for horizon
   
    carbonstore for cs
   
    add time
   
    optimize
   
    CI pass

commit 6deba93b73cd06df6330ca36c90e40060eb2c381
Author: xubo245 <xubo29@...>
Date:   2018-08-30T02:18:28Z

    Rebase to carbondata master
    1.TODO: S3 AK/SK configure

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/91/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8162/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6473/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8164/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/93/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6475/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8166/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6477/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/95/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8186/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/115/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8191/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    retest this please


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/120/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8192/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/121/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8201/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/130/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2673: [WIP] Test Carbonstore

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2673
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8243/



---
12