[jira] [Created] (CARBONDATA-1277) Dictionary generation failure if there is failure in closing output steam in HDFS

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (CARBONDATA-1277) Dictionary generation failure if there is failure in closing output steam in HDFS

Akash R Nilugal (Jira)
Manish Gupta created CARBONDATA-1277:
----------------------------------------

             Summary: Dictionary generation failure if there is failure in closing output steam in HDFS
                 Key: CARBONDATA-1277
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1277
             Project: CarbonData
          Issue Type: Bug
            Reporter: Manish Gupta
            Assignee: Manish Gupta
             Fix For: 1.2.0


If there is any failure while closing the output stream of dictionary file in HDFS then on next data load, update or insert into operation dictionary generation fails. This is because we open the dictionary file in append mode and when we try to get the output stream for that file HDFS throws an exception that Lease is already acquired by some other client. Exception trace as below.
java.io.IOException: Failed to APPEND_FILE /user/hive/warehouse/carbon.store/test/t12/Metadata/010497af-1833-4804-a7ec-849ab7b9bf10.dictmeta for DFSClient_NONMAPREDUCE_-1904161438_159 on 172.168.100.212 because lease recovery is in progress. Try again later.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2901)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2655)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2968)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2937)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:776)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)