[jira] [Commented] (CARBONDATA-37) Support Date/Time format for Timestamp columns to be defined at column level

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607199#comment-15607199 ]

ASF GitHub Bot commented on CARBONDATA-37:
------------------------------------------

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/219#discussion_r85039488
 
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/surrogatekeysgenerator/csvbased/CarbonCSVBasedSeqGenStep.java ---
    @@ -470,6 +474,36 @@ public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws K
                   break;
               }
             }
    +        HashMap<String, String> dateformatsHashMap = new HashMap<String, String>();
    +        if (meta.dateFormat != null) {
    +          String[] dateformats = meta.dateFormat.split(",");
    +          for (String dateFormat:dateformats) {
    +            String[] dateFormatSplits = dateFormat.split(":", 2);
    +            dateformatsHashMap.put(dateFormatSplits[0],dateFormatSplits[1]);
    +            // TODO  verify the dateFormatSplits is valid or not
    +          }
    +        }
    +        directDictionaryGenerators =
    +                new DirectDictionaryGenerator[meta.getDimensionColumnIds().length];
    +        for (int i = 0; i < meta.getDimensionColumnIds().length; i++) {
    +          ColumnSchemaDetails columnSchemaDetails = columnSchemaDetailsWrapper.get(
    +                  meta.getDimensionColumnIds()[i]);
    +          if (columnSchemaDetails.isDirectDictionary()) {
    +            if (dateformatsHashMap.containsKey(columnSchemaDetails.getColumnName())) {
    +              directDictionaryGenerators[i] =
    +                      DirectDictionaryKeyGeneratorFactory.getDirectDictionaryGenerator(
    +                              columnSchemaDetails.getColumnType(),
    +                              dateformatsHashMap.get(columnSchemaDetails.getColumnName()));
    +            } else {
    +              String dateFormat = CarbonProperties.getInstance()
    +                      .getProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
    +                              CarbonCommonConstants.CARBON_TIMESTAMP_DEFAULT_FORMAT);
    +              directDictionaryGenerators[i] =
    +                      DirectDictionaryKeyGeneratorFactory.getDirectDictionaryGenerator(
    +                              columnSchemaDetails.getColumnType(), dateFormat);
    --- End diff --
   
    1. move out CarbonProperties.getInstance().getProperty  from for loop
    2. for defaut  dataformat, use  method getDirectDictionaryGenerator(DataType dataType)

> Support Date/Time format for Timestamp columns to be defined at column level
> ----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-37
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-37
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: Vimal Das Kammath
>            Assignee: Lionx
>
> Carbon support defining the Date/Time format. But the configuration for the same is present in carbon.properties and hence is global for all tables.
> This global configuration for timestamp format cannot support scenarios where different tables or different Timestamp columns in the same table.
> Suggest to provide option in the create table DDL itself to define the format for each Timestamp column. Also provide defaults so that users can create table with Timestamp columns without having to always define the Date/Time format.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)