[GitHub] carbondata pull request #2848: [CARBONDATA-3036] Cache Columns And Refresh T...

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2848: [CARBONDATA-3036] Cache Columns And Refresh T...

qiuchenjian-2
GitHub user manishnalla1994 opened a pull request:

    https://github.com/apache/carbondata/pull/2848

    [CARBONDATA-3036] Cache Columns And Refresh Table Isuue Fix

    Refresh Table Issue : Refresh Table command acting in case sensitive manner.
    Cache Columns Issue : Results inconsistent when cache is set but min/max exceeds. Columns are dictionary excluded.
   
    Fix 1 : Path for carbon file was been taken as whatever table name given in the query(Lowercase/Uppercase). Changed it to lowercase.
    Fix 2 : MinMaxFlag array was not set according to the columns to be cached giving inconsistent results. Changed it according to the min/max values array for whatever columns given in Cache Columns only.
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [x] Testing done
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishnalla1994/carbondata RefreshAndCacheColumnsFix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2848.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2848
   
----
commit 7158960c750cf6ed7243e1c7c4bbc44fe158326c
Author: Manish Nalla <manishnalla1994@...>
Date:   2018-10-24T05:45:15Z

    CacheAndRefreshIsuueFix

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1195/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9249/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/983/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user manishnalla1994 commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    retest this please



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2848: [CARBONDATA-3036] Cache Columns And Refresh T...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2848#discussion_r227731562
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/allqueries/TestQueryWithColumnMetCacheAndCacheLevelProperty.scala ---
    @@ -282,6 +285,48 @@ class TestQueryWithColumnMetCacheAndCacheLevelProperty extends QueryTest with Be
         sql("drop table if exists alter_add_column_min_max")
       }
     
    +  test("Test For Cache set but Min/Max exceeds") {
    +    sql("DROP TABLE IF EXISTS carbonCache")
    +    sql(
    +      s"""
    +         | CREATE TABLE carbonCache (
    +         | name STRING,
    +         | age STRING,
    +         | desc STRING
    +         | )
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES('COLUMN_META_CACHE'='name,desc')
    +       """.stripMargin)
    +    sql(
    +      "INSERT INTO carbonCache values('Manish Nalla','24'," +
    +      "'gvsahgvsahjvcsahjgvavacavkjvaskjvsahgsvagkjvkjgvsackjg')")
    +    checkAnswer(sql(
    +      "SELECT count(*) FROM carbonCache where " +
    +      "desc='gvsahgvsahjvcsahjgvavacavkjvaskjvsahgsvagkjvkjgvsackjg'"),
    +      Row(1))
    +  }
    +
    +  test("Cache Blocklet Level testing") {
    --- End diff --
   
    1. give proper name for the test case
    2. Drop the table before and after completion of each test case
    3. Add the property for setting max byte count before the start of each test case and in after all set the property back to default


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/991/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1204/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9257/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1017/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1230/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9282/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2848: [CARBONDATA-3036] Cache Columns And Refresh Table Is...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/2848
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2848: [CARBONDATA-3036] Cache Columns And Refresh T...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2848


---