[jira] [Created] (CARBONDATA-2875) Two different threads are overwriting the same carbondatafile

[Akash R Nilugal (Jira)]

Supritha created CARBONDATA-2875:
------------------------------------

             Summary: Two different threads are overwriting the same carbondatafile
                 Key: CARBONDATA-2875
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2875
             Project: CarbonData
          Issue Type: Bug
          Components: data-load
    Affects Versions: 1.5.0
         Environment: Spark 2.2
            Reporter: Supritha


Steps :

sparS.sql("create external table if not exists thread_x stored by 'carbondata' location 'D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread'").show();
 sparS.sql("select count(*) from thread_x").show(); ----> 300000 record count
 
 try{
 System.out.println("count called after write");
 Thread t1 = new Thread(new Trial(sparS));
 Thread t2 = new Thread(new Trial(sparS));
 //Each thread is holding 2 records each
 t1.setName("T1");
 t2.setName("T2");
 t1.start();
 
 Thread.sleep(10);
 t2.start();
 t1.join();
 t2.join();

sparS.sql("select count(*) from thread_x").show();
System.out.println("counrt called after load");

 

Actual Issue : sparS.sql("select count(*) from thread_x").show(); -----> 300002

Which implies Two different threads are overwriting the same carbondatafile.

Expected : 300004 record count should be displayed. 

ie A carbonfile has to be created for each thread.

 

From logs its is seen that two different threads are overwriting the same carbondatafile

Line 298: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
 Line 302: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
 Line 374: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
 Line 378: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)