Hi all,
when I run the following script: scala> cc.sql(s"load data inpath 'hdfs://master:9000/carbondata/sample.csv' into table test_table") it turns out that: WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): java.lang.RuntimeException: Dictionary file name is locked for updation. Please try after some time what I have done are: 1.in carbon.properties, set carbon.lock.type=HDFSLOCK 2.send carbon.properties & spark-defaults.conf to all nodes of the clusters if any of you have any idea, looking forward to your replay, thx~ |
Hi,
can you please find *"HDFS lock path"* string in executor log and let me know the complete log message. -Regards Kumar Vishal On Tue, Dec 27, 2016 at 12:45 PM, 251469031 <[hidden email]> wrote: > Hi all, > > > when I run the following script: > scala> cc.sql(s"load data inpath 'hdfs://master:9000/carbondata/sample.csv' > into table test_table") > > > it turns out that: > WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): > java.lang.RuntimeException: Dictionary file name is locked for updation. > Please try after some time > > > what I have done are: > 1.in carbon.properties, set carbon.lock.type=HDFSLOCK > 2.send carbon.properties & spark-defaults.conf to all nodes of the clusters > > > if any of you have any idea, looking forward to your replay, thx~
kumar vishal
|
Hi Kumar,
thx to your repley, the full logs is as follows: 16/12/27 12:30:17 INFO locks.HdfsFileLock: Executor task launch worker-0 HDFS lock path:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 1 blocks 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Started 1 remote fetches in 1 ms 16/12/27 12:30:32 ERROR rdd.CarbonGlobalDictionaryGenerateRDD: Executor task launch worker-0 java.lang.RuntimeException: Dictionary file name is locked for updation. Please try after some time at scala.sys.package$.error(package.scala:27) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.<init>(CarbonGlobalDictionaryRDD.scala:364) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) as u see, the lock file path is:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock ------------------ 原始邮件 ------------------ 发件人: "Kumar Vishal";<[hidden email]>; 发送时间: 2016年12月27日(星期二) 下午3:25 收件人: "dev"<[hidden email]>; 主题: Re: Dictionary file is locked for updation Hi, can you please find *"HDFS lock path"* string in executor log and let me know the complete log message. -Regards Kumar Vishal On Tue, Dec 27, 2016 at 12:45 PM, 251469031 <[hidden email]> wrote: > Hi all, > > > when I run the following script: > scala> cc.sql(s"load data inpath 'hdfs://master:9000/carbondata/sample.csv' > into table test_table") > > > it turns out that: > WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): > java.lang.RuntimeException: Dictionary file name is locked for updation. > Please try after some time > > > what I have done are: > 1.in carbon.properties, set carbon.lock.type=HDFSLOCK > 2.send carbon.properties & spark-defaults.conf to all nodes of the clusters > > > if any of you have any idea, looking forward to your replay, thx~ |
Hi,
It seems the store path location is taking default location. Did you set the store location properly? Which spark version you are using? Regards, Ravindra On Tue, Dec 27, 2016, 1:38 PM 251469031 <[hidden email]> wrote: > Hi Kumar, > > > thx to your repley, the full logs is as follows: > > > 16/12/27 12:30:17 INFO locks.HdfsFileLock: Executor task launch worker-0 > HDFS lock > path:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Getting 1 > non-empty blocks out of 1 blocks > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Started 1 > remote fetches in 1 ms > 16/12/27 12:30:32 ERROR rdd.CarbonGlobalDictionaryGenerateRDD: Executor > task launch worker-0 > java.lang.RuntimeException: Dictionary file name is locked for updation. > Please try after some time > at scala.sys.package$.error(package.scala:27) > at > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.<init>(CarbonGlobalDictionaryRDD.scala:364) > at > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at > org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) > at org.apache.spark.scheduler.Task.run(Task.scala:89) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > > as u see, the lock file path > is:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > > > > > ------------------ 原始邮件 ------------------ > 发件人: "Kumar Vishal";<[hidden email]>; > 发送时间: 2016年12月27日(星期二) 下午3:25 > 收件人: "dev"<[hidden email]>; > > 主题: Re: Dictionary file is locked for updation > > > > Hi, > can you please find *"HDFS lock path"* string in executor log and let me > know the complete log message. > > -Regards > Kumar Vishal > > On Tue, Dec 27, 2016 at 12:45 PM, 251469031 <[hidden email]> wrote: > > > Hi all, > > > > > > when I run the following script: > > scala> cc.sql(s"load data inpath > 'hdfs://master:9000/carbondata/sample.csv' > > into table test_table") > > > > > > it turns out that: > > WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): > > java.lang.RuntimeException: Dictionary file name is locked for updation. > > Please try after some time > > > > > > what I have done are: > > 1.in carbon.properties, set carbon.lock.type=HDFSLOCK > > 2.send carbon.properties & spark-defaults.conf to all nodes of the > clusters > > > > > > if any of you have any idea, looking forward to your replay, thx~ |
I'm using spark 1.6.0, and the carbondata is the latest master branch in the github.
my carbon.properties is configured as : carbon.ddl.base.hdfs.url=hdfs://master:9000/carbondata/data carbon.badRecords.location=/opt/Carbon/Spark/badrecords carbon.kettle.home=/opt/spark-1.6.0/carbonlib/carbonplugins carbon.lock.type=HDFSLOCK .... my spark-default.conf is configured as : spark.master spark://master:7077 spark.yarn.dist.files /opt/spark-1.6.0/conf/carbon.properties spark.yarn.dist.archives /opt/spark-1.6.0/carbonlib/carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar spark.executor.extraJavaOptions -Dcarbon.properties.filepath=carbon.properties #spark.executor.extraClassPath /opt/spark-1.6.0/carbonlib/carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar #spark.driver.extraClassPath /opt/spark-1.6.0/carbonlib/carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar spark.driver.extraJavaOptions -Dcarbon.properties.filepath=/opt/spark-1.6.0/conf/carbon.properties carbon.kettle.home /opt/spark-1.6.0/carbonlib/carbonplugins ------------------ 原始邮件 ------------------ 发件人: "Ravindra Pesala";<[hidden email]>; 发送时间: 2016年12月27日(星期二) 下午4:15 收件人: "dev"<[hidden email]>; 主题: Re: Dictionary file is locked for updation Hi, It seems the store path location is taking default location. Did you set the store location properly? Which spark version you are using? Regards, Ravindra On Tue, Dec 27, 2016, 1:38 PM 251469031 <[hidden email]> wrote: > Hi Kumar, > > > thx to your repley, the full logs is as follows: > > > 16/12/27 12:30:17 INFO locks.HdfsFileLock: Executor task launch worker-0 > HDFS lock > path:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Getting 1 > non-empty blocks out of 1 blocks > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Started 1 > remote fetches in 1 ms > 16/12/27 12:30:32 ERROR rdd.CarbonGlobalDictionaryGenerateRDD: Executor > task launch worker-0 > java.lang.RuntimeException: Dictionary file name is locked for updation. > Please try after some time > at scala.sys.package$.error(package.scala:27) > at > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.<init>(CarbonGlobalDictionaryRDD.scala:364) > at > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at > org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) > at org.apache.spark.scheduler.Task.run(Task.scala:89) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > > as u see, the lock file path > is:hdfs://master:9000../carbon.store/default/test_table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > > > > > ------------------ 原始邮件 ------------------ > 发件人: "Kumar Vishal";<[hidden email]>; > 发送时间: 2016年12月27日(星期二) 下午3:25 > 收件人: "dev"<[hidden email]>; > > 主题: Re: Dictionary file is locked for updation > > > > Hi, > can you please find *"HDFS lock path"* string in executor log and let me > know the complete log message. > > -Regards > Kumar Vishal > > On Tue, Dec 27, 2016 at 12:45 PM, 251469031 <[hidden email]> wrote: > > > Hi all, > > > > > > when I run the following script: > > scala> cc.sql(s"load data inpath > 'hdfs://master:9000/carbondata/sample.csv' > > into table test_table") > > > > > > it turns out that: > > WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): > > java.lang.RuntimeException: Dictionary file name is locked for updation. > > Please try after some time > > > > > > what I have done are: > > 1.in carbon.properties, set carbon.lock.type=HDFSLOCK > > 2.send carbon.properties & spark-defaults.conf to all nodes of the > clusters > > > > > > if any of you have any idea, looking forward to your replay, thx~ |
Please try to add carbon.storelocation to carbon.properties file.
e.g. carbon.storelocation=hdfs://master:9000/carbondata/store You can have a look the following guide and pay attention to carbon.properties file. https://cwiki.apache.org/confluence/display/CARBONDATA/Cluster+deployment+guide
Best Regards
David Cai |
I'm sorry, carbon.storelocation has been configured in my cluster. I didn't copy it. the configuration is:
carbon.storelocation=hdfs://master:9000/carbondata ------------------ Original ------------------ From: "QiangCai";<[hidden email]>; Date: Tue, Dec 27, 2016 05:29 PM To: "dev"<[hidden email]>; Subject: Re: 回复: Dictionary file is locked for updation Please try to add carbon.storelocation to carbon.properties file. e.g. carbon.storelocation=hdfs://master:9000/carbondata/store You can have a look the following guide and pay attention to carbon.properties file. https://cwiki.apache.org/confluence/display/CARBONDATA/Cluster+deployment+guide -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Dictionary-file-is-locked-for-updation-tp5076p5090.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com. |
please correct the path of carbon.properties file.
spark.executor.extraJavaOptions -Dcarbon.properties.filepath=carbon.properties
Best Regards
David Cai |
thx QiangCai, the problem is solved.
so, maybe it's better to correct the document at https://cwiki.apache.org/confluence/display/CARBONDATA/Cluster+deployment+guide, change the value of spark.executor.extraJavaOptions from -Dcarbon.properties.filepath=carbon.properties to -Dcarbon.properties.filepath=<YOUR_SPARK_HOME_PATH>"/conf/carbon.properties ------------------ Original ------------------ From: "QiangCai";<[hidden email]>; Date: Tue, Dec 27, 2016 05:40 PM To: "dev"<[hidden email]>; Subject: Re: 回复: Dictionary file is locked for updation please correct the path of carbon.properties file. spark.executor.extraJavaOptions -Dcarbon.properties.filepath=carbon.properties -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Dictionary-file-is-locked-for-updation-tp5076p5092.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com. |
In reply to this post by 李寅威
Hi,
Can you set the store location in carbon properties. *carbon.storelocation=hdfs://master:9000/carbondata/<store_location>* -Regards Kumar Vishal On Tue, Dec 27, 2016 at 4:27 PM, 251469031 <[hidden email]> wrote: > I'm using spark 1.6.0, and the carbondata is the latest master branch in > the github. > > > my carbon.properties is configured as : > > > carbon.ddl.base.hdfs.url=hdfs://master:9000/carbondata/data > carbon.badRecords.location=/opt/Carbon/Spark/badrecords > carbon.kettle.home=/opt/spark-1.6.0/carbonlib/carbonplugins > > carbon.lock.type=HDFSLOCK > .... > > > my spark-default.conf is configured as : > > > spark.master spark://master:7077 > spark.yarn.dist.files > /opt/spark-1.6.0/conf/carbon.properties > spark.yarn.dist.archives /opt/spark-1.6.0/carbonlib/ > carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar > spark.executor.extraJavaOptions > -Dcarbon.properties.filepath=carbon.properties > #spark.executor.extraClassPath /opt/spark-1.6.0/carbonlib/ > carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar > #spark.driver.extraClassPath /opt/spark-1.6.0/carbonlib/ > carbondata_2.10-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar > spark.driver.extraJavaOptions > -Dcarbon.properties.filepath=/opt/spark-1.6.0/conf/carbon.properties > carbon.kettle.home /opt/spark-1.6.0/carbonlib/ > carbonplugins > > > > > > ------------------ 原始邮件 ------------------ > 发件人: "Ravindra Pesala";<[hidden email]>; > 发送时间: 2016年12月27日(星期二) 下午4:15 > 收件人: "dev"<[hidden email]>; > > 主题: Re: Dictionary file is locked for updation > > > > Hi, > > It seems the store path location is taking default location. Did you set > the store location properly? Which spark version you are using? > > Regards, > Ravindra > > On Tue, Dec 27, 2016, 1:38 PM 251469031 <[hidden email]> wrote: > > > Hi Kumar, > > > > > > thx to your repley, the full logs is as follows: > > > > > > 16/12/27 12:30:17 INFO locks.HdfsFileLock: Executor task launch worker-0 > > HDFS lock > > path:hdfs://master:9000../carbon.store/default/test_ > table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Getting 1 > > non-empty blocks out of 1 blocks > > 16/12/27 12:30:17 INFO storage.ShuffleBlockFetcherIterator: Started 1 > > remote fetches in 1 ms > > 16/12/27 12:30:32 ERROR rdd.CarbonGlobalDictionaryGenerateRDD: Executor > > task launch worker-0 > > java.lang.RuntimeException: Dictionary file name is locked for updation. > > Please try after some time > > at scala.sys.package$.error(package.scala:27) > > at > > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerate > RDD$$anon$1.<init>(CarbonGlobalDictionaryRDD.scala:364) > > at > > org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerate > RDD.compute(CarbonGlobalDictionaryRDD.scala:302) > > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD. > scala:306) > > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > > at > > org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) > > at org.apache.spark.scheduler.Task.run(Task.scala:89) > > at > > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > as u see, the lock file path > > is:hdfs://master:9000../carbon.store/default/test_ > table/2e9b7efa-2934-463a-9280-ff50c5129268.lock > > > > > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: "Kumar Vishal";<[hidden email]>; > > 发送时间: 2016年12月27日(星期二) 下午3:25 > > 收件人: "dev"<[hidden email]>; > > > > 主题: Re: Dictionary file is locked for updation > > > > > > > > Hi, > > can you please find *"HDFS lock path"* string in executor log and let me > > know the complete log message. > > > > -Regards > > Kumar Vishal > > > > On Tue, Dec 27, 2016 at 12:45 PM, 251469031 <[hidden email]> wrote: > > > > > Hi all, > > > > > > > > > when I run the following script: > > > scala> cc.sql(s"load data inpath > > 'hdfs://master:9000/carbondata/sample.csv' > > > into table test_table") > > > > > > > > > it turns out that: > > > WARN 27-12 12:37:58,044 - Lost task 1.3 in stage 2.0 (TID 13, slave1): > > > java.lang.RuntimeException: Dictionary file name is locked for > updation. > > > Please try after some time > > > > > > > > > what I have done are: > > > 1.in carbon.properties, set carbon.lock.type=HDFSLOCK > > > 2.send carbon.properties & spark-defaults.conf to all nodes of the > > clusters > > > > > > > > > if any of you have any idea, looking forward to your replay, thx~ >
kumar vishal
|
Administrator
|
In reply to this post by 李寅威
Hi
Updated ,thanks for you pointed out the issue. Regards Liang
|
Hi all,
raised a jira issue : https://issues.apache.org/jira/browse/CARBONDATA-585 On the execution of the following query : LOAD DATA inpath 'hdfs://localhost:54310/csv/test.csv' INTO table employee options('DELIMITER'=',', 'FILEHEADER'='id, firstname'); the table schema is a following : -----------------------------+ col_name data_type comment -----------------------------+ id bigint firstname string -----------------------------+ The load gets successful at times but we also end up often with the following error : Dictionary file is locked for Updation. Following below are the logs : AUDIT 02-01 18:17:07,009 - [knoldus][pallavi][Thread-110]Dataload failure for default.employee. Please check the logs INFO 02-01 18:17:07,020 - pool-30-thread-1 Successfully deleted the lock file /tmp/default/employee/meta.lock INFO 02-01 18:17:07,022 - Table MetaData Unlocked Successfully after data load ERROR 02-01 18:17:07,022 - Error executing query, currentState RUNNING, org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 5, 192.168.2.188): java.lang.RuntimeException: Dictionary file firstname is locked for updation. Please try after some time at scala.sys.package$.error(package.scala:27) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD$$anon$1.<init>(CarbonGlobalDictionaryRDD.scala:364) at org.apache.carbondata.spark.rdd.CarbonGlobalDictionaryGenerateRDD.compute(CarbonGlobalDictionaryRDD.scala:302) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929) at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:927) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111) at org.apache.spark.rdd.RDD.withScope(RDD.scala:316) at org.apache.spark.rdd.RDD.collect(RDD.scala:926) at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.generateGlobalDictionary(GlobalDictionaryUtil.scala:769) On Tue, Dec 27, 2016 at 8:10 PM, Liang Chen <[hidden email]> wrote: > Hi > > Updated ,thanks for you pointed out the issue. > > Regards > Liang > > > 李寅威 wrote > > thx QiangCai, the problem is solved. > > > > > > so, maybe it's better to correct the document at > > https://cwiki.apache.org/confluence/display/CARBONDATA/ > Cluster+deployment+guide, > > change the value of spark.executor.extraJavaOptions > > > > > > from > > -Dcarbon.properties.filepath=carbon.properties > > > > > > to > > -Dcarbon.properties.filepath= > > <YOUR_SPARK_HOME_PATH> > > "/conf/carbon.properties > > > > > > > > > > > > ------------------ Original ------------------ > > From: "QiangCai";< > > > qiangcai@ > > > >; > > Date: Tue, Dec 27, 2016 05:40 PM > > To: "dev"< > > > dev@.apache > > > >; > > > > Subject: Re: 回复: Dictionary file is locked for updation > > > > > > > > please correct the path of carbon.properties file. > > > > spark.executor.extraJavaOptions > > -Dcarbon.properties.filepath=carbon.properties > > > > > > > > > > > > -- > > View this message in context: > > http://apache-carbondata-mailing-list-archive.1130556. > n5.nabble.com/Dictionary-file-is-locked-for-updation-tp5076p5092.html > > Sent from the Apache CarbonData Mailing List archive mailing list archive > > at Nabble.com. > > > > > > -- > View this message in context: http://apache-carbondata- > mailing-list-archive.1130556.n5.nabble.com/Dictionary-file- > is-locked-for-updation-tp5076p5103.html > Sent from the Apache CarbonData Mailing List archive mailing list archive > at Nabble.com. > -- Regards | Pallavi Singh Software Consultant Knoldus Software LLP [hidden email] +91-9911235949 |
Free forum by Nabble | Edit this page |