Login  Register

[jira] [Created] (CARBONDATA-824) Null pointer Exception display to user while performance Testing

classic Classic list List threaded Threaded
2 messages Options Options
Embed post
Permalink
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

[jira] [Created] (CARBONDATA-824) Null pointer Exception display to user while performance Testing

Akash R Nilugal (Jira)
14274 posts
Vinod Rohilla created CARBONDATA-824:
----------------------------------------

             Summary: Null pointer Exception display to user while performance Testing
                 Key: CARBONDATA-824
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-824
             Project: CarbonData
          Issue Type: Bug
          Components: data-query
    Affects Versions: 1.0.1-incubating
         Environment: SPARK 2.1
            Reporter: Vinod Rohilla


Displays null pointer exception to the user while select Query.

Steps to reproduces:

1: Create table:

CREATE TABLE oscon_new_1 (ACTIVE_AREA_ID String, ACTIVE_CHECK_DY String, ACTIVE_CHECK_HOUR String, ACTIVE_CHECK_MM String, ACTIVE_CHECK_TIME String, ACTIVE_CHECK_YR String, ACTIVE_CITY String, ACTIVE_COUNTRY String, ACTIVE_DISTRICT String, ACTIVE_EMUI_VERSION String, ACTIVE_FIRMWARE_VER String, ACTIVE_NETWORK String, ACTIVE_OS_VERSION String, ACTIVE_PROVINCE String, BOM String, CHECK_DATE String, CHECK_DY String, CHECK_HOUR String, CHECK_MM String, CHECK_YR String, CUST_ADDRESS_ID String, CUST_AGE String, CUST_BIRTH_COUNTRY String, CUST_BIRTH_DY String, CUST_BIRTH_MM String, CUST_BIRTH_YR String, CUST_BUY_POTENTIAL String, CUST_CITY String, CUST_STATE String, CUST_COUNTRY String, CUST_COUNTY String, CUST_EMAIL_ADDR String, CUST_LAST_RVW_DATE TIMESTAMP, CUST_FIRST_NAME String, CUST_ID String, CUST_JOB_TITLE String, CUST_LAST_NAME String, CUST_LOGIN String, CUST_NICK_NAME String, CUST_PRFRD_FLG String, CUST_SEX String, CUST_STREET_NAME String, CUST_STREET_NO String, CUST_SUITE_NO String, CUST_ZIP String, DELIVERY_CITY String, DELIVERY_STATE String, DELIVERY_COUNTRY String, DELIVERY_DISTRICT String, DELIVERY_PROVINCE String, DEVICE_NAME String, INSIDE_NAME String, ITM_BRAND String, ITM_BRAND_ID String, ITM_CATEGORY String, ITM_CATEGORY_ID String, ITM_CLASS String, ITM_CLASS_ID String, ITM_COLOR String, ITM_CONTAINER String, ITM_FORMULATION String, ITM_MANAGER_ID String, ITM_MANUFACT String, ITM_MANUFACT_ID String, ITM_ID String, ITM_NAME String, ITM_REC_END_DATE String, ITM_REC_START_DATE String, LATEST_AREAID String, LATEST_CHECK_DY String, LATEST_CHECK_HOUR String, LATEST_CHECK_MM String, LATEST_CHECK_TIME String, LATEST_CHECK_YR String, LATEST_CITY String, LATEST_COUNTRY String, LATEST_DISTRICT String, LATEST_EMUI_VERSION String, LATEST_FIRMWARE_VER String, LATEST_NETWORK String, LATEST_OS_VERSION String, LATEST_PROVINCE String, OL_ORDER_DATE String, OL_ORDER_NO INT, OL_RET_ORDER_NO String, OL_RET_DATE String, OL_SITE String, OL_SITE_DESC String, PACKING_DATE String, PACKING_DY String, PACKING_HOUR String, PACKING_LIST_NO String, PACKING_MM String, PACKING_YR String, PRMTION_ID String, PRMTION_NAME String, PRM_CHANNEL_CAT String, PRM_CHANNEL_DEMO String, PRM_CHANNEL_DETAILS String, PRM_CHANNEL_DMAIL String, PRM_CHANNEL_EMAIL String, PRM_CHANNEL_EVENT String, PRM_CHANNEL_PRESS String, PRM_CHANNEL_RADIO String, PRM_CHANNEL_TV String, PRM_DSCNT_ACTIVE String, PRM_END_DATE String, PRM_PURPOSE String, PRM_START_DATE String, PRODUCT_ID String, PROD_BAR_CODE String, PROD_BRAND_NAME String, PRODUCT_NAME String, PRODUCT_MODEL String, PROD_MODEL_ID String, PROD_COLOR String, PROD_SHELL_COLOR String, PROD_CPU_CLOCK String, PROD_IMAGE String, PROD_LIVE String, PROD_LOC String, PROD_LONG_DESC String, PROD_RAM String, PROD_ROM String, PROD_SERIES String, PROD_SHORT_DESC String, PROD_THUMB String, PROD_UNQ_DEVICE_ADDR String, PROD_UNQ_MDL_ID String, PROD_UPDATE_DATE String, PROD_UQ_UUID String, SHP_CARRIER String, SHP_CODE String, SHP_CONTRACT String, SHP_MODE_ID String, SHP_MODE String, STR_ORDER_DATE String, STR_ORDER_NO String, TRACKING_NO String, WH_CITY String, WH_COUNTRY String, WH_COUNTY String, WH_ID String, WH_NAME String, WH_STATE String, WH_STREET_NAME String, WH_STREET_NO String, WH_STREET_TYPE String, WH_SUITE_NO String, WH_ZIP String, CUST_DEP_COUNT DOUBLE, CUST_VEHICLE_COUNT DOUBLE, CUST_ADDRESS_CNT DOUBLE, CUST_CRNT_CDEMO_CNT DOUBLE, CUST_CRNT_HDEMO_CNT DOUBLE, CUST_CRNT_ADDR_DM DOUBLE, CUST_FIRST_SHIPTO_CNT DOUBLE, CUST_FIRST_SALES_CNT DOUBLE, CUST_GMT_OFFSET DOUBLE, CUST_DEMO_CNT DOUBLE, CUST_INCOME DOUBLE, PROD_UNLIMITED INT, PROD_OFF_PRICE DOUBLE, PROD_UNITS INT, TOTAL_PRD_COST DOUBLE, TOTAL_PRD_DISC DOUBLE, PROD_WEIGHT DOUBLE, REG_UNIT_PRICE DOUBLE, EXTENDED_AMT DOUBLE, UNIT_PRICE_DSCNT_PCT DOUBLE, DSCNT_AMT DOUBLE, PROD_STD_CST DOUBLE, TOTAL_TX_AMT DOUBLE, FREIGHT_CHRG DOUBLE, WAITING_PERIOD DOUBLE, DELIVERY_PERIOD DOUBLE, ITM_CRNT_PRICE DOUBLE, ITM_UNITS DOUBLE, ITM_WSLE_CST DOUBLE, ITM_SIZE DOUBLE, PRM_CST DOUBLE, PRM_RESPONSE_TARGET DOUBLE, PRM_ITM_DM DOUBLE, SHP_MODE_CNT DOUBLE, WH_GMT_OFFSET DOUBLE, WH_SQ_FT DOUBLE, STR_ORD_QTY DOUBLE, STR_WSLE_CST DOUBLE, STR_LIST_PRICE DOUBLE, STR_SALES_PRICE DOUBLE, STR_EXT_DSCNT_AMT DOUBLE, STR_EXT_SALES_PRICE DOUBLE, STR_EXT_WSLE_CST DOUBLE, STR_EXT_LIST_PRICE DOUBLE, STR_EXT_TX DOUBLE, STR_COUPON_AMT DOUBLE, STR_NET_PAID DOUBLE, STR_NET_PAID_INC_TX DOUBLE, STR_NET_PRFT DOUBLE, STR_SOLD_YR_CNT DOUBLE, STR_SOLD_MM_CNT DOUBLE, STR_SOLD_ITM_CNT DOUBLE, STR_TOTAL_CUST_CNT DOUBLE, STR_AREA_CNT DOUBLE, STR_DEMO_CNT DOUBLE, STR_OFFER_CNT DOUBLE, STR_PRM_CNT DOUBLE, STR_TICKET_CNT DOUBLE, STR_NET_PRFT_DM_A DOUBLE, STR_NET_PRFT_DM_B DOUBLE, STR_NET_PRFT_DM_C DOUBLE, STR_NET_PRFT_DM_D DOUBLE, STR_NET_PRFT_DM_E DOUBLE, STR_RET_STR_ID DOUBLE, STR_RET_REASON_CNT DOUBLE, STR_RET_TICKET_NO DOUBLE, STR_RTRN_QTY DOUBLE, STR_RTRN_AMT DOUBLE, STR_RTRN_TX DOUBLE, STR_RTRN_AMT_INC_TX DOUBLE, STR_RET_FEE DOUBLE, STR_RTRN_SHIP_CST DOUBLE, STR_RFNDD_CSH DOUBLE, STR_REVERSED_CHRG DOUBLE, STR_STR_CREDIT DOUBLE, STR_RET_NET_LOSS DOUBLE, STR_RTRNED_YR_CNT DOUBLE, STR_RTRN_MM_CNT DOUBLE, STR_RET_ITM_CNT DOUBLE, STR_RET_CUST_CNT DOUBLE, STR_RET_AREA_CNT DOUBLE, STR_RET_OFFER_CNT DOUBLE, STR_RET_PRM_CNT DOUBLE, STR_RET_NET_LOSS_DM_A DOUBLE, STR_RET_NET_LOSS_DM_B DOUBLE, STR_RET_NET_LOSS_DM_C DOUBLE, STR_RET_NET_LOSS_DM_D DOUBLE, OL_ORD_QTY DOUBLE, OL_WSLE_CST DOUBLE, OL_LIST_PRICE DOUBLE, OL_SALES_PRICE DOUBLE, OL_EXT_DSCNT_AMT DOUBLE, OL_EXT_SALES_PRICE DOUBLE, OL_EXT_WSLE_CST DOUBLE, OL_EXT_LIST_PRICE DOUBLE, OL_EXT_TX DOUBLE, OL_COUPON_AMT DOUBLE, OL_EXT_SHIP_CST DOUBLE, OL_NET_PAID DOUBLE, OL_NET_PAID_INC_TX DOUBLE, OL_NET_PAID_INC_SHIP DOUBLE, OL_NET_PAID_INC_SHIP_TX DOUBLE, OL_NET_PRFT DOUBLE, OL_SOLD_YR_CNT DOUBLE, OL_SOLD_MM_CNT DOUBLE, OL_SHIP_DATE_CNT DOUBLE, OL_ITM_CNT DOUBLE, OL_BILL_CUST_CNT DOUBLE, OL_BILL_AREA_CNT DOUBLE, OL_BILL_DEMO_CNT DOUBLE, OL_BILL_OFFER_CNT DOUBLE, OL_SHIP_CUST_CNT DOUBLE, OL_SHIP_AREA_CNT DOUBLE, OL_SHIP_DEMO_CNT DOUBLE, OL_SHIP_OFFER_CNT DOUBLE, OL_WEB_PAGE_CNT DOUBLE, OL_WEB_SITE_CNT DOUBLE, OL_SHIP_MODE_CNT DOUBLE, OL_WH_CNT DOUBLE, OL_PRM_CNT DOUBLE, OL_NET_PRFT_DM_A DOUBLE, OL_NET_PRFT_DM_B DOUBLE, OL_NET_PRFT_DM_C DOUBLE, OL_NET_PRFT_DM_D DOUBLE, OL_RET_RTRN_QTY DOUBLE, OL_RTRN_AMT DOUBLE, OL_RTRN_TX DOUBLE, OL_RTRN_AMT_INC_TX DOUBLE, OL_RET_FEE DOUBLE, OL_RTRN_SHIP_CST DOUBLE, OL_RFNDD_CSH DOUBLE, OL_REVERSED_CHRG DOUBLE, OL_ACCOUNT_CREDIT DOUBLE, OL_RTRNED_YR_CNT DOUBLE, OL_RTRNED_MM_CNT DOUBLE, OL_RTRITM_CNT DOUBLE, OL_RFNDD_CUST_CNT DOUBLE, OL_RFNDD_AREA_CNT DOUBLE, OL_RFNDD_DEMO_CNT DOUBLE, OL_RFNDD_OFFER_CNT DOUBLE, OL_RTRNING_CUST_CNT DOUBLE, OL_RTRNING_AREA_CNT DOUBLE, OL_RTRNING_DEMO_CNT DOUBLE, OL_RTRNING_OFFER_CNT DOUBLE, OL_RTRWEB_PAGE_CNT DOUBLE, OL_REASON_CNT DOUBLE, OL_NET_LOSS DOUBLE, OL_NET_LOSS_DM_A DOUBLE, OL_NET_LOSS_DM_B DOUBLE, OL_NET_LOSS_DM_C DOUBLE) STORED BY 'org.apache.carbondata.format';


2: Perform Select Query.
TC_010,""select * from  oscon_new_1  where CUST_ID = """"ID00000000015"""" and CUST_CITY=""""CC015"""" and CUST_LAST_RVW_DATE between """"2011-02-17 00:00:00"""" and """"2011-02-18 00:00:00""""""

TC_011,""select * from   oscon_new_1  where PROD_COLOR =""""GOLD"""" and ACTIVE_EMUI_VERSION  like""""%73"""" limit 5000""

TC_022,""SELECT *  from  oscon_new_1  where CUST_PRFRD_FLG=""""Y"""" and PROD_BRAND_NAME = """"LG"""" and PROD_COLOR = """"BLACK"""" and CUST_LAST_RVW_DATE = """"2011-07-02 00:00:00"""" and CUST_COUNTRY = """"CC009"""" and product_name = """"LG KF311 phone"""" ""

TC_019,"select * from  oscon_new_1  where CUST_CITY='CC008' and CUST_PRFRD_FLG='Y' and CUST_LAST_RVW_DATE between '2011-02-04 00:00:00' and '2011-02-05 00:00:00'"

TC_008,""select * from  oscon_new_1  where CUST_CITY=""""CC015"""" and CUST_PRFRD_FLG=""""Y"""" and CUST_LAST_RVW_DATE between """"2011-02-02 00:00:00"""" and """"2011-02-03 00:00:00""""""

Error :

WARN TaskSetManager: Lost task 0.0 in stage 69.0 (TID 24007, hadoop-slave-7, executor 8): java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:136)
        at org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:50)
        at org.apache.carbondata.core.scan.processor.impl.DataBlockIteratorImpl.next(DataBlockIteratorImpl.java:32)
        at org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:50)
        at org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
        at org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:31)
        at org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.<init>(ChunkRowIterator.java:41)
        at org.apache.carbondata.hadoop.CarbonRecordReader.initialize(CarbonRecordReader.java:79)
        at org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.scala:204)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:99)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.getNextScannedResult(AbstractDataBlockIterator.java:146)
        at org.apache.carbondata.core.scan.processor.AbstractDataBlockIterator.updateScanner(AbstractDataBlockIterator.java:124)
        ... 28 more


Note : Data load file (CSV) is 32 GB so which can not be upload on the Jira.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Re: [jira] [Created] (CARBONDATA-824) Null pointer Exception display to user while performance Testing

ZhuWilliam
23 posts
First, can you try change the create table statement ends with STORED BY
'carbondata' instead of STORED BY 'org.apache.carbondata.format';
Second , can you give some sample data instead of trying to upload 32GB CSV
file.