Apache CarbonData Dev Mailing List archive › Apache CarbonData JIRA issues

[jira] [Updated] (CARBONDATA-1869) (Carbon1.3.0 - Spark 2.2) Null pointer exception thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX

classic Classic list List threaded Threaded
♦
♦
 1 message Options
Akash R Nilugal (Jira)
Reply | Threaded
Open this post in threaded view
♦
♦
|

[jira] [Updated] (CARBONDATA-1869) (Carbon1.3.0 - Spark 2.2) Null pointer exception thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX

Akash R Nilugal (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhatchayani updated CARBONDATA-1869:
------------------------------------
    Description:
Steps -
From beeline terminal a table is created with with table properties having dictionary exclude or NO_INVERTED_INDEX-
create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('DICTIONARY_EXCLUDE'='a1');   or
create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('NO_INVERTED_INDEX'='a1');

  From 3 concurrent beeline terminals the below sequence of insert into select and select queries are executed 120 times.
  insert into test select 2147483647;
  select * from test;
  select count(*) from test;
  select a1 from test;
  select round(a1),bround(a1),floor(a1),ceil(a1),rand(),exp(a1),ln(a1),log10(a1),log2(1),log(a1),pow(a1,a1),sqrt(a1),bin(a1),pmod(a1,a1),sin(a1),asin(a1),cos(a1),tan(a1),atan(a1),degrees(a1),radians(a1),positive(a1),negative(a1),sign(a1),factorial(a1),cbrt(a1) from test;

【Expected Output】：The insert into select query should be successful and the null pointer exception should not be thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX.

【Actual Output】：Null pointer exception thrown when concurrent load and select queries executed with table properties having dictionary exclude or NO_INVERTED_INDEX
  0: jdbc:hive2://10.18.98.136:23040> insert into test select 2147483647;
Error: java.lang.NullPointerException (state=,code=0)




*+{color:red}Stacktrace:{color}+*
java.lang.NullPointerException
        at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.delete(AbstractDFSCarbonFile.java:152)
        at org.apache.carbondata.processing.util.DeleteLoadFolders.physicalFactAndMeasureMetadataDeletion(DeleteLoadFolders.java:90)
        at org.apache.carbondata.processing.util.DeleteLoadFolders.deleteLoadFoldersFromFileSystem(DeleteLoadFolders.java:134)
        at org.apache.carbondata.spark.rdd.DataManagementFunc$.deleteLoadsAndUpdateMetadata(DataManagementFunc.scala:187)
        at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:296)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:362)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:193)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.run(CarbonLoadDataCommand.scala:65)
        at org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:43)
        at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:71)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
        at org.apache.spark.sql.Dataset.<init>(Dataset.scala:182)
        at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
        at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:623)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:691)

  was:
Steps -
From beeline terminal a table is created with with table properties having dictionary exclude or NO_INVERTED_INDEX-
create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('DICTIONARY_EXCLUDE'='a1');   or
create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('NO_INVERTED_INDEX'='a1');

  From 3 concurrent beeline terminals the below sequence of insert into select and select queries are executed 120 times.
  insert into test select 2147483647;
  select * from test;
  select count(*) from test;
  select a1 from test;
  select round(a1),bround(a1),floor(a1),ceil(a1),rand(),exp(a1),ln(a1),log10(a1),log2(1),log(a1),pow(a1,a1),sqrt(a1),bin(a1),pmod(a1,a1),sin(a1),asin(a1),cos(a1),tan(a1),atan(a1),degrees(a1),radians(a1),positive(a1),negative(a1),sign(a1),factorial(a1),cbrt(a1) from test;

【Expected Output】：The insert into select query should be successful and the null pointer exception should not be thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX.

【Actual Output】：Null pointer exception thrown when concurrent load and select queries executed with table properties having dictionary exclude or NO_INVERTED_INDEX
  0: jdbc:hive2://10.18.98.136:23040> insert into test select 2147483647;
Error: java.lang.NullPointerException (state=,code=0)




Stacktrace:
java.lang.NullPointerException
        at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.delete(AbstractDFSCarbonFile.java:152)
        at org.apache.carbondata.processing.util.DeleteLoadFolders.physicalFactAndMeasureMetadataDeletion(DeleteLoadFolders.java:90)
        at org.apache.carbondata.processing.util.DeleteLoadFolders.deleteLoadFoldersFromFileSystem(DeleteLoadFolders.java:134)
        at org.apache.carbondata.spark.rdd.DataManagementFunc$.deleteLoadsAndUpdateMetadata(DataManagementFunc.scala:187)
        at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:296)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:362)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:193)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.run(CarbonLoadDataCommand.scala:65)
        at org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:43)
        at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:71)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
        at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
        at org.apache.spark.sql.Dataset.<init>(Dataset.scala:182)
        at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
        at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:623)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:691)


> (Carbon1.3.0 - Spark 2.2) Null pointer exception thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-1869
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1869
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>    Affects Versions: 1.3.0
>         Environment: 3 node ant cluster
>            Reporter: Chetan Bhat
>            Assignee: dhatchayani
>              Labels: DFX
>          Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Steps -
> From beeline terminal a table is created with with table properties having dictionary exclude or NO_INVERTED_INDEX-
> create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('DICTIONARY_EXCLUDE'='a1');   or
> create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('NO_INVERTED_INDEX'='a1');
>   From 3 concurrent beeline terminals the below sequence of insert into select and select queries are executed 120 times.
>   insert into test select 2147483647;
>   select * from test;
>   select count(*) from test;
>   select a1 from test;
>   select round(a1),bround(a1),floor(a1),ceil(a1),rand(),exp(a1),ln(a1),log10(a1),log2(1),log(a1),pow(a1,a1),sqrt(a1),bin(a1),pmod(a1,a1),sin(a1),asin(a1),cos(a1),tan(a1),atan(a1),degrees(a1),radians(a1),positive(a1),negative(a1),sign(a1),factorial(a1),cbrt(a1) from test;
> 【Expected Output】：The insert into select query should be successful and the null pointer exception should not be thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX.
> 【Actual Output】：Null pointer exception thrown when concurrent load and select queries executed with table properties having dictionary exclude or NO_INVERTED_INDEX
>   0: jdbc:hive2://10.18.98.136:23040> insert into test select 2147483647;
> Error: java.lang.NullPointerException (state=,code=0)
> *+{color:red}Stacktrace:{color}+*
> java.lang.NullPointerException
> at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.delete(AbstractDFSCarbonFile.java:152)
> at org.apache.carbondata.processing.util.DeleteLoadFolders.physicalFactAndMeasureMetadataDeletion(DeleteLoadFolders.java:90)
> at org.apache.carbondata.processing.util.DeleteLoadFolders.deleteLoadFoldersFromFileSystem(DeleteLoadFolders.java:134)
> at org.apache.carbondata.spark.rdd.DataManagementFunc$.deleteLoadsAndUpdateMetadata(DataManagementFunc.scala:187)
> at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:296)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:362)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:193)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.run(CarbonLoadDataCommand.scala:65)
> at org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:43)
> at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:71)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
> at org.apache.spark.sql.Dataset.<init>(Dataset.scala:182)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:623)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:691)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Free forum by Nabble Edit this page