[jira] [Closed] (CARBONDATA-4049) Sometimes refresh table fails with error "table not found in database" error

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chetan Bhat closed CARBONDATA-4049.
-----------------------------------
    Resolution: Won't Fix

Issue is identified as that of refresh which is handled by Spark. As issue is not a carbon issue its closed.

> Sometimes refresh table fails with error "table not found in database" error
> ----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4049
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4049
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-query
>    Affects Versions: 2.1.0
>         Environment: Spark 2.4.5
>            Reporter: Chetan Bhat
>            Priority: Minor
>
> In Carbon 2.1 version user creates a database.
> user copies a old version store such as 1.6.1 to HDFS folder of the database in the In Carbon 2.1 version
> In Spark-SQL or beeline the user accesses the database using the use db command.
> Refresh table command is executed on the old version store table and then the subsequent operations on the table are performed.
> Next refresh table command is tried to be executed on another old version store table .
>  
> Issue : Sometimes refresh table fails with error "table not found in database" error.
> spark-sql> refresh table brinjal_deleteseg;
> *Error in query: Table or view 'brinjal_deleteseg' not found in database '1_6_1';*
>  
> **Log -
> 2020-11-12 18:55:46,922 | INFO  | [main] | Created broadcast 171 from broadCastHadoopConf at CarbonRDD.scala:58 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,922 | INFO  | [main] | Created broadcast 171 from broadCastHadoopConf at CarbonRDD.scala:58 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,924 | INFO  | [main] | Pushed Filters:  | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,939 | INFO  | [main] | Distributed Index server is enabled for 1_6_1.brinjal_update | org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12 18:55:46,939 | INFO  | [main] | Started block pruning ... | org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:526)2020-11-12 18:55:46,940 | INFO  | [main] | Distributed Index server is enabled for 1_6_1.brinjal_update | org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12 18:55:46,945 | INFO  | [main] | Successfully Created directory: hdfs://hacluster/tmp/indexservertmp/4b6353d4-65d7-4856-b3cd-b3bc11d15c55 | org.apache.carbondata.core.util.CarbonUtil.createTempFolderForIndexServer(CarbonUtil.java:3273)2020-11-12 18:55:46,945 | INFO  | [main] | Temp folder path for Query ID: 4b6353d4-65d7-4856-b3cd-b3bc11d15c55 is org.apache.carbondata.core.datastore.filesystem.HDFSCarbonFile@b8f2e1bf | org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:57)2020-11-12 18:55:46,946 | ERROR | [main] | Configured port for index server is not a valid number | org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1779)java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:542) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1777) at org.apache.carbondata.indexserver.IndexServer$.serverPort$lzycompute(IndexServer.scala:88) at org.apache.carbondata.indexserver.IndexServer$.serverPort(IndexServer.scala:88) at org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:312) at org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:301) at org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:83) at org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:59) at org.apache.carbondata.spark.util.CarbonScalaUtil$.logTime(CarbonScalaUtil.scala:769) at org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:58) at org.apache.carbondata.core.index.IndexUtil.executeIndexJob(IndexUtil.java:304) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:431) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:532) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:477) at org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:356) at org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:204) at org.apache.carbondata.spark.rdd.CarbonScanRDD.internalGetPartitions(CarbonScanRDD.scala:159) at org.apache.carbondata.spark.rdd.CarbonRDD.getPartitions(CarbonRDD.scala:68) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126) at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:990) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:385) at org.apache.spark.rdd.RDD.collect(RDD.scala:989) at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:299) at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:326) at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:128) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64) at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:371) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:274) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)2020-11-12 18:55:46,949 | ERROR | [main] | Exception occurred while getting splits using index server. Initiating Fall back to embedded mode | org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:438)java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:542) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.carbondata.core.util.CarbonProperties.getIndexServerPort(CarbonProperties.java:1777) at org.apache.carbondata.indexserver.IndexServer$.serverPort$lzycompute(IndexServer.scala:88) at org.apache.carbondata.indexserver.IndexServer$.serverPort(IndexServer.scala:88) at org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:312) at org.apache.carbondata.indexserver.IndexServer$.getClient(IndexServer.scala:301) at org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:83) at org.apache.carbondata.indexserver.DistributedIndexJob$$anonfun$1.apply(IndexJobs.scala:59) at org.apache.carbondata.spark.util.CarbonScalaUtil$.logTime(CarbonScalaUtil.scala:769) at org.apache.carbondata.indexserver.DistributedIndexJob.execute(IndexJobs.scala:58) at org.apache.carbondata.core.index.IndexUtil.executeIndexJob(IndexUtil.java:304) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getDistributedSplit(CarbonInputFormat.java:431) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:532) at org.apache.carbondata.hadoop.api.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:477) at org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:356) at org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:204) at org.apache.carbondata.spark.rdd.CarbonScanRDD.internalGetPartitions(CarbonScanRDD.scala:159) at org.apache.carbondata.spark.rdd.CarbonRDD.getPartitions(CarbonRDD.scala:68) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:269) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126) at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:990) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:385) at org.apache.spark.rdd.RDD.collect(RDD.scala:989) at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:299) at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:326) at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:128) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver$$anonfun$run$1.apply(SparkSQLDriver.scala:64) at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:371) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:274) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)2020-11-12 18:55:46,951 | INFO  | [main] | Block broadcast_172 stored as values in memory (estimated size 370.9 KB, free 909.1 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,960 | INFO  | [main] | Block broadcast_172_piece0 stored as bytes in memory (estimated size 29.4 KB, free 909.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,961 | INFO  | [dispatcher-event-loop-1] | Added broadcast_172_piece0 in memory on vm1:43460 (size: 29.4 KB, free: 912.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,961 | INFO  | [main] | Created broadcast 172 from broadCastHadoopConf at CarbonRDD.scala:58 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,962 | INFO  | [main] | Distributed Index server is enabled for 1_6_1.brinjal_update | org.apache.carbondata.core.util.CarbonProperties.isDistributedPruningEnabled(CarbonProperties.java:1742)2020-11-12 18:55:46,966 | INFO  | [main] | Starting job: collect at IndexServer.scala:178 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,967 | INFO  | [dag-scheduler-event-loop] | Got job 87 (collect at IndexServer.scala:178) with 1 output partitions | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,967 | INFO  | [dag-scheduler-event-loop] | Final stage: ResultStage 83 (collect at IndexServer.scala:178) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,967 | INFO  | [dag-scheduler-event-loop] | Parents of final stage: List() | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,967 | INFO  | [dag-scheduler-event-loop] | Missing parents: List() | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,967 | INFO  | [dag-scheduler-event-loop] | Submitting ResultStage 83 (DistributedPruneRDD[249] at RDD at CarbonRDD.scala:38), which has no missing parents | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,968 | INFO  | [dag-scheduler-event-loop] | Block broadcast_173 stored as values in memory (estimated size 17.7 KB, free 909.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,969 | INFO  | [dag-scheduler-event-loop] | Block broadcast_173_piece0 stored as bytes in memory (estimated size 8.7 KB, free 909.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,970 | INFO  | [dispatcher-event-loop-2] | Added broadcast_173_piece0 in memory on vm1:43460 (size: 8.7 KB, free: 912.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,970 | INFO  | [dag-scheduler-event-loop] | Created broadcast 173 from broadcast at DAGScheduler.scala:1163 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,970 | INFO  | [dag-scheduler-event-loop] | Submitting 1 missing tasks from ResultStage 83 (DistributedPruneRDD[249] at RDD at CarbonRDD.scala:38) (first 15 tasks are for partitions Vector(0)) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,970 | INFO  | [dag-scheduler-event-loop] | Adding task set 83.0 with 1 tasks | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,971 | INFO  | [dispatcher-event-loop-5] | Starting task 0.0 in stage 83.0 (TID 464, localhost, executor driver, partition 0, PROCESS_LOCAL, 9449 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,971 | INFO  | [Executor task launch worker for task 464] | Running task 0.0 in stage 83.0 (TID 464) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,974 | INFO  | [Executor task launch worker for task 464] | Value for carbon.max.executor.threads.for.block.pruning is 4 | org.apache.carbondata.core.util.CarbonProperties.getNumOfThreadsForExecutorPruning(CarbonProperties.java:1809)2020-11-12 18:55:46,995 | INFO  | [IndexPruningPool_1605178546974] | Constructing new SegmentProperties for table: 1_6_1_brinjal_update. Current size of segment properties holder list is: 2 | org.apache.carbondata.core.datastore.block.SegmentPropertiesAndSchemaHolder.addSegmentProperties(SegmentPropertiesAndSchemaHolder.java:115)2020-11-12 18:55:46,997 | INFO  | [IndexPruningPool_1605178546974] | Removed entry from InMemory lru cache :: hdfs://hacluster/user/sparkhive/warehouse/1_6_1.db/brinjal_update/Fact/Part0/Segment_5/5_1605178541880.carbonindexmerge | org.apache.carbondata.core.cache.CarbonLRUCache.removeKey(CarbonLRUCache.java:189)2020-11-12 18:55:46,997 | INFO  | [Executor task launch worker for task 464] | Time taken to collect 1 blocklets : 23 | org.apache.carbondata.indexserver.DistributedPruneRDD.internalCompute(DistributedPruneRDD.scala:118)2020-11-12 18:55:46,998 | INFO  | [Executor task launch worker for task 464] | Finished task 0.0 in stage 83.0 (TID 464). 2475 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,999 | INFO  | [task-result-getter-3] | Finished task 0.0 in stage 83.0 (TID 464) in 27 ms on localhost (executor driver) (1/1) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,999 | INFO  | [task-result-getter-3] | Removed TaskSet 83.0, whose tasks have all completed, from pool  | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,999 | INFO  | [dag-scheduler-event-loop] | ResultStage 83 (collect at IndexServer.scala:178) finished in 0.031 s | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:46,999 | INFO  | [main] | Job 87 finished: collect at IndexServer.scala:178, took 0.032684 s | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,001 | INFO  | [main] | Block broadcast_174 stored as values in memory (estimated size 370.9 KB, free 908.6 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,010 | INFO  | [main] | Block broadcast_174_piece0 stored as bytes in memory (estimated size 29.4 KB, free 908.6 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,011 | INFO  | [dispatcher-event-loop-7] | Added broadcast_174_piece0 in memory on vm1:43460 (size: 29.4 KB, free: 912.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,011 | INFO  | [main] | Created broadcast 174 from broadCastHadoopConf at CarbonRDD.scala:58 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,016 | INFO  | [main] | Starting job: collect at IndexServer.scala:205 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,017 | INFO  | [main] | Job 88 finished: collect at IndexServer.scala:205, took 0.000032 s | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,017 | INFO  | [main] | Finished block pruning ... | org.apache.carbondata.hadoop.api.CarbonInputFormat.getPrunedBlocklets(CarbonInputFormat.java:622)2020-11-12 18:55:47,018 | INFO  | [main] | Planning scan with bin packing, max size: 4194304 bytes, open cost is considered as scanning 4194304 bytes. | org.apache.carbondata.spark.rdd.CarbonScanRDD.distributeColumnarSplits(CarbonScanRDD.scala:366)2020-11-12 18:55:47,018 | INFO  | [main] |  Identified no.of.blocks: 5, no.of.tasks: 5, no.of.nodes: 0, parallelism: 8        | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,022 | INFO  | [main] | Starting job: processCmd at CliDriver.java:376 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,023 | INFO  | [dag-scheduler-event-loop] | Got job 89 (processCmd at CliDriver.java:376) with 5 output partitions | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,023 | INFO  | [dag-scheduler-event-loop] | Final stage: ResultStage 84 (processCmd at CliDriver.java:376) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,023 | INFO  | [dag-scheduler-event-loop] | Parents of final stage: List() | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,023 | INFO  | [dag-scheduler-event-loop] | Missing parents: List() | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,023 | INFO  | [dag-scheduler-event-loop] | Submitting ResultStage 84 (MapPartitionsRDD[248] at processCmd at CliDriver.java:376), which has no missing parents | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,025 | INFO  | [dag-scheduler-event-loop] | Block broadcast_175 stored as values in memory (estimated size 24.3 KB, free 908.6 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,026 | INFO  | [dag-scheduler-event-loop] | Block broadcast_175_piece0 stored as bytes in memory (estimated size 11.6 KB, free 908.6 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,026 | INFO  | [dispatcher-event-loop-6] | Added broadcast_175_piece0 in memory on vm1:43460 (size: 11.6 KB, free: 912.0 MB) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,026 | INFO  | [dag-scheduler-event-loop] | Created broadcast 175 from broadcast at DAGScheduler.scala:1163 | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,027 | INFO  | [dag-scheduler-event-loop] | Submitting 5 missing tasks from ResultStage 84 (MapPartitionsRDD[248] at processCmd at CliDriver.java:376) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4)) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,027 | INFO  | [dag-scheduler-event-loop] | Adding task set 84.0 with 5 tasks | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,028 | INFO  | [dispatcher-event-loop-1] | Starting task 0.0 in stage 84.0 (TID 465, localhost, executor driver, partition 0, ANY, 9185 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,028 | INFO  | [dispatcher-event-loop-1] | Starting task 1.0 in stage 84.0 (TID 466, localhost, executor driver, partition 1, ANY, 9185 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,028 | INFO  | [dispatcher-event-loop-1] | Starting task 2.0 in stage 84.0 (TID 467, localhost, executor driver, partition 2, ANY, 9185 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [dispatcher-event-loop-1] | Starting task 3.0 in stage 84.0 (TID 468, localhost, executor driver, partition 3, ANY, 9185 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [dispatcher-event-loop-1] | Starting task 4.0 in stage 84.0 (TID 469, localhost, executor driver, partition 4, ANY, 9185 bytes) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [Executor task launch worker for task 466] | Running task 1.0 in stage 84.0 (TID 466) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [Executor task launch worker for task 467] | Running task 2.0 in stage 84.0 (TID 467) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [Executor task launch worker for task 469] | Running task 4.0 in stage 84.0 (TID 469) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [Executor task launch worker for task 468] | Running task 3.0 in stage 84.0 (TID 468) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,029 | INFO  | [Executor task launch worker for task 465] | Running task 0.0 in stage 84.0 (TID 465) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,033 | INFO  | [Executor task launch worker for task 469] | Projection Columns: [imei, amsize, channelsid, activecountry, activecity, productiondate, deliverydate, gamepointid, deviceinformationid, deliverycharge] | org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12 18:55:47,033 | INFO  | [Executor task launch worker for task 467] | Projection Columns: [imei, amsize, channelsid, activecountry, activecity, productiondate, deliverydate, gamepointid, deviceinformationid, deliverycharge] | org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12 18:55:47,033 | INFO  | [Executor task launch worker for task 466] | Projection Columns: [imei, amsize, channelsid, activecountry, activecity, productiondate, deliverydate, gamepointid, deviceinformationid, deliverycharge] | org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12 18:55:47,033 | INFO  | [Executor task launch worker for task 468] | Projection Columns: [imei, amsize, channelsid, activecountry, activecity, productiondate, deliverydate, gamepointid, deviceinformationid, deliverycharge] | org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12 18:55:47,033 | INFO  | [Executor task launch worker for task 465] | Projection Columns: [imei, amsize, channelsid, activecountry, activecity, productiondate, deliverydate, gamepointid, deviceinformationid, deliverycharge] | org.apache.carbondata.core.scan.model.QueryModelBuilder.projectColumns(QueryModelBuilder.java:94)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 466] | Query will be executed on table: brinjal_update | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 467] | Query will be executed on table: brinjal_update | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 469] | Query will be executed on table: brinjal_update | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 466] | Query prefetch is: true | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 465] | Query will be executed on table: brinjal_update | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 469] | Query prefetch is: true | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 465] | Query prefetch is: true | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 468] | Query will be executed on table: brinjal_update | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.initQuery(AbstractQueryExecutor.java:122)2020-11-12 18:55:47,034 | INFO  | [Executor task launch worker for task 467] | Query prefetch is: true | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12 18:55:47,035 | INFO  | [Executor task launch worker for task 468] | Query prefetch is: true | org.apache.carbondata.core.scan.executor.impl.AbstractQueryExecutor.getBlockExecutionInfoForBlock(AbstractQueryExecutor.java:479)2020-11-12 18:55:47,036 | INFO  | [Executor task launch worker for task 466] | Vector based dictionary collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12 18:55:47,036 | INFO  | [Executor task launch worker for task 466] | Direct page-wise vector fill collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 469] | Vector based dictionary collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 469] | Direct page-wise vector fill collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 465] | Vector based dictionary collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 465] | Direct page-wise vector fill collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 468] | Vector based dictionary collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 468] | Direct page-wise vector fill collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12 18:55:47,037 | INFO  | [Executor task launch worker for task 467] | Vector based dictionary collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.ResultCollectorFactory.getScannedResultCollector(ResultCollectorFactory.java:78)2020-11-12 18:55:47,038 | INFO  | [Executor task launch worker for task 467] | Direct page-wise vector fill collector is used to scan and collect the data | org.apache.carbondata.core.scan.collector.impl.DictionaryBasedVectorResultCollector.<init>(DictionaryBasedVectorResultCollector.java:73)2020-11-12 18:55:47,050 | INFO  | [Executor task launch worker for task 465] | Total off-heap working memory used after task 645a72eb-8d06-489b-940d-f7ca1e901bc7 is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, f251027c-707b-4df1-b35f-00eb74c8d77f, 5233fa5c-5173-4a9e-a8bf-8c024dac1509, a05de0f7-398e-45a5-a91d-a4e1045e9a98 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,050 | INFO  | [Executor task launch worker for task 467] | Total off-heap working memory used after task a05de0f7-398e-45a5-a91d-a4e1045e9a98 is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, f251027c-707b-4df1-b35f-00eb74c8d77f, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,050 | INFO  | [Executor task launch worker for task 469] | Total off-heap working memory used after task f251027c-707b-4df1-b35f-00eb74c8d77f is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,050 | INFO  | [Executor task launch worker for task 467] | Total off-heap working memory used after task 11861831-3ae3-4dd7-98ce-352f4416e4d5 is 128073. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,050 | INFO  | [Executor task launch worker for task 469] | Total off-heap working memory used after task c4193df5-ceb8-4ff9-8844-0d9c42f989d8 is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 467] | Total off-heap working memory used after task 88e4dbd6-c19e-43e6-bc9b-42dfefc8378c is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 469] | Total off-heap working memory used after task f75746ea-783d-4098-91d5-1a93fda81f04 is 128045. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 465] | Total off-heap working memory used after task ae4ac11d-7a43-426d-8578-ac77b6ddf74b is 128090. Current running tasks are 7a7a3012-c819-4e4c-9f08-54dd890850e4, 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 466] | Total off-heap working memory used after task 7a7a3012-c819-4e4c-9f08-54dd890850e4 is 128090. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 465] | Total off-heap working memory used after task fb7aa505-1163-4fe2-936e-0de1e4f78271 is 128090. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 466] | Total off-heap working memory used after task 27cb623e-3b1c-4f49-adaa-c11c19d766af is 128000. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 466] | Total off-heap working memory used after task b2d72c4c-7d5d-4030-9f79-54219d56474d is 0. Current running tasks are 5233fa5c-5173-4a9e-a8bf-8c024dac1509 | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,052 | INFO  | [Executor task launch worker for task 465] | Finished task 0.0 in stage 84.0 (TID 465). 2766 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 469] | Finished task 4.0 in stage 84.0 (TID 469). 2198 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,051 | INFO  | [Executor task launch worker for task 467] | Finished task 2.0 in stage 84.0 (TID 467). 2444 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,052 | INFO  | [Executor task launch worker for task 466] | Finished task 1.0 in stage 84.0 (TID 466). 2524 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,052 | INFO  | [Executor task launch worker for task 468] | Total off-heap working memory used after task 5233fa5c-5173-4a9e-a8bf-8c024dac1509 is 0. Current running tasks are  | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,052 | INFO  | [Executor task launch worker for task 468] | Total off-heap working memory used after task 4ea15806-df6e-4377-9cea-84f07586fe02 is 0. Current running tasks are  | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,052 | INFO  | [task-result-getter-2] | Finished task 0.0 in stage 84.0 (TID 465) in 25 ms on localhost (executor driver) (1/5) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,052 | INFO  | [Executor task launch worker for task 468] | Total off-heap working memory used after task 1086a9e4-a644-4340-8215-b8c3d936e32d is 0. Current running tasks are  | org.apache.carbondata.core.memory.UnsafeMemoryManager.freeMemoryAll(UnsafeMemoryManager.java:179)2020-11-12 18:55:47,053 | INFO  | [task-result-getter-1] | Finished task 4.0 in stage 84.0 (TID 469) in 24 ms on localhost (executor driver) (2/5) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,053 | INFO  | [Executor task launch worker for task 468] | Finished task 3.0 in stage 84.0 (TID 468). 2380 bytes result sent to driver | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,053 | INFO  | [task-result-getter-0] | Finished task 2.0 in stage 84.0 (TID 467) in 25 ms on localhost (executor driver) (3/5) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,054 | INFO  | [task-result-getter-3] | Finished task 1.0 in stage 84.0 (TID 466) in 26 ms on localhost (executor driver) (4/5) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,054 | INFO  | [task-result-getter-2] | Finished task 3.0 in stage 84.0 (TID 468) in 26 ms on localhost (executor driver) (5/5) | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,054 | INFO  | [task-result-getter-2] | Removed TaskSet 84.0, whose tasks have all completed, from pool  | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,054 | INFO  | [dag-scheduler-event-loop] | ResultStage 84 (processCmd at CliDriver.java:376) finished in 0.030 s | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,055 | INFO  | [main] | Job 89 finished: processCmd at CliDriver.java:376, took 0.032288 s | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)2020-11-12 18:55:47,071 | INFO  | [main] | Time taken: 0.344 seconds, Fetched 534 row(s) | org.apache.hadoop.hive.ql.session.SessionState$LogHelper.printInfo(SessionState.java:951)2020-11-12 18:55:47,086 | AUDIT | [main] | \{"time":"November 12, 2020 6:55:47 PM CST","username":"root","opName":"SET","opId":"3304241283944785","opStatus":"START"} | org.apache.carbondata.processing.util.Auditor.logOperationStart(Auditor.java:74)2020-11-12 18:55:47,086 | INFO  | [main] | The key carbon.enable.index.server with value false added in the session param | org.apache.carbondata.core.util.SessionParams.addProperty(SessionParams.java:102)2020-11-12 18:55:47,087 | AUDIT | [main] | \{"time":"November 12, 2020 6:55:47 PM CST","username":"root","opName":"SET","opId":"3304241283944785","opStatus":"SUCCESS","opTime":"1 ms","table":"NA","extraInfo":{}} | org.apache.carbondata.processing.util.Auditor.logOperationEnd(Auditor.java:97)2020-11-12 18:55:47,094 | INFO  | [main] | Time taken: 0.016 seconds, Fetched 1 row(s) | org.apache.hadoop.hive.ql.session.SessionState$LogHelper.printInfo(SessionState.java:951)2020-11-12 18:58:04,451 | AUDIT | [main] | \{"time":"November 12, 2020 6:58:04 PM CST","username":"root","opName":"REFRESH TABLE","opId":"3304378649144145","opStatus":"START"} | org.apache.carbondata.processing.util.Auditor.logOperationStart(Auditor.java:74)2020-11-12 18:58:04,452 | INFO  | [main] | 0: get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,452 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,455 | INFO  | [main] | 0: get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,455 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,458 | INFO  | [main] | 0: get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,458 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,460 | INFO  | [main] | 0: get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,460 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,462 | INFO  | [main] | 0: get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,462 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,465 | INFO  | [main] | 0: get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,465 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_database: 1_6_1 | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,467 | INFO  | [main] | 0: get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logInfo(HiveMetaStore.java:746)2020-11-12 18:58:04,467 | INFO  | [main] | ugi=root ip=unknown-ip-addr cmd=get_table : db=1_6_1 tbl=brinjal_deleteseg | org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.logAuditEvent(HiveMetaStore.java:371)2020-11-12 18:58:04,472 | AUDIT | [main] | \{"time":"November 12, 2020 6:58:04 PM CST","username":"root","opName":"REFRESH TABLE","opId":"3304378649144145","opStatus":"FAILED","opTime":"21 ms","table":"1_6_1.brinjal_deleteseg","extraInfo":{"Exception":"org.apache.spark.sql.catalyst.analysis.NoSuchTableException","Message":"Table or view 'brinjal_deleteseg' not found in database '1_6_1';"}} | org.apache.carbondata.processing.util.Auditor.logOperationEnd(Auditor.java:97)

--
This message was sent by Atlassian Jira
(v8.3.4#803005)