[jira] [Commented] (CARBONDATA-304) Load data failure when set table_blocksize=2048

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CARBONDATA-304) Load data failure when set table_blocksize=2048

Akash R Nilugal (Jira)

    [ https://issues.apache.org/jira/browse/CARBONDATA-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568825#comment-15568825 ]

ASF GitHub Bot commented on CARBONDATA-304:
-------------------------------------------

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/227#discussion_r83012391
 
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java ---
    @@ -197,8 +197,9 @@ public AbstractFactDataWriter(String storeLocation, int measureCount, int mdKeyL
         blockIndexInfoList = new ArrayList<>();
         // get max file size;
         CarbonProperties propInstance = CarbonProperties.getInstance();
    -    this.fileSizeInBytes = blocksize * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR
    -        * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR * 1L;
    +    // if blocksize=2048, then 2048*1024*1024 will beyond the range of Int
    +    this.fileSizeInBytes = 1L * blocksize * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR
    --- End diff --
   
    instead of multiple by `1L`, you can just convert `blocksize` to `(long)blocksize` instead


> Load data failure when set table_blocksize=2048
> -----------------------------------------------
>
>                 Key: CARBONDATA-304
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-304
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Gin-zhj
>            Assignee: Gin-zhj
>
> First ,create a table with table_blocksize=2048
> CREATE TABLE IF NOT EXISTS t3 (ID Int, date Timestamp, country String, name String, phonetype String, serialname String, salary Int) STORED BY 'carbondata' TBLPROPERTIES('table_blocksize'='2048');
> Then load data, failure and catch exception:
> org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException: Problem while copying file from local store to carbon store



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)