[jira] [Updated] (CARBONDATA-1535) Complex data type ARRAY<STRING> do not support auto merge/compaction

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

xubo245 updated CARBONDATA-1535:
--------------------------------
    Description:
When carbon.enable.auto.load.merge was set as true, the table that has complex data type ARRAY<STRING>  will happen error when compaction(auto merge).
 
 Complex data type ARRAY<STRING>  doesn't support some operations as follow:

# auto merge
#  sql("alter table array_table compact 'major'")
#  sql("alter table array_table compact 'minor'")


{code:java}
java.lang.IndexOutOfBoundsException: 1
        at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
        at scala.collection.immutable.List.apply(List.scala:84)
        at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
        at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)

{code}


  was:
When carbon.enable.auto.load.merge was set as true, the table that has complex data type ARRAY<STRING>  will happen error when compaction(auto merge).
 
 as follow:

# auto merge
#  sql("alter table array_table compact 'major'")
#  sql("alter table array_table compact 'minor'")


{code:java}
java.lang.IndexOutOfBoundsException: 1
        at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
        at scala.collection.immutable.List.apply(List.scala:84)
        at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
        at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
        at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)

{code}

> Complex data type ARRAY<STRING>  do not support auto merge/compaction
> ---------------------------------------------------------------------
>
>                 Key: CARBONDATA-1535
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1535
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: xubo245
>   Original Estimate: 720h
>  Remaining Estimate: 720h
>
> When carbon.enable.auto.load.merge was set as true, the table that has complex data type ARRAY<STRING>  will happen error when compaction(auto merge).
>  
>  Complex data type ARRAY<STRING>  doesn't support some operations as follow:
> # auto merge
> #  sql("alter table array_table compact 'major'")
> #  sql("alter table array_table compact 'minor'")
> {code:java}
> java.lang.IndexOutOfBoundsException: 1
> at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
> at scala.collection.immutable.List.apply(List.scala:84)
> at scala.collection.convert.Wrappers$SeqWrapper.get(Wrappers.scala:66)
> at org.apache.carbondata.core.datastore.block.SegmentProperties.readAllComplexTypeChildren(SegmentProperties.java:445)
> at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:390)
> at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
> at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
> at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
> at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
> {code}

--
This message was sent by Atlassian JIRA
(v6.4.14#64029)