how to load the dictionary file when loading data.

classic Classic list List threaded Threaded
3 messages Options
ytx
Reply | Threaded
Open this post in threaded view
|

how to load the dictionary file when loading data.

ytx
I just want to know how to load the dictionary file when loading data.
example:load data inpath '/person.txt' into table person
OPTIONS('DELIMITER'='|','QUOTECHAR'='"','FILEHEADER'='id,name,birthday,start_time,work_locations,course','COMPLEX_DELIMITER_LEVEL_1'='$','ALL_DICTIONARY_PATH'='/opt/dictionary/allcolumn.dictionary');
When I add content in the allcolumn.dictionary file, the load fails, but the
content is empty and loaded successfully. So I wonder if it's my
allcolumn.dictionary is wrong? AND if it is added ALL_DICTIONARY_PATH what
effect?
Reply | Threaded
Open this post in threaded view
|

Re: how to load the dictionary file when loading data.

David CaiQiang
Hi,
    You can have a look AllDictionaryExample in spark module.

    A example of dict file as following:

2,usa
2,china
1,2015/7/26
1,2015/7/23
1,2015/7/30
3,aaa3
3,aaa10

   The line format is   "<column index in csv file header, start from 0>,<dict vlaue>"

Best regards
David QiangCai
Best Regards
David Cai
Reply | Threaded
Open this post in threaded view
|

Re: how to load the dictionary file when loading data.

bhavya411
In reply to this post by ytx
Hi ,

Can you send me the the content that you are using for Dictionary file as
it will be easier to see what is wrong there.

Regards
Bhavya

On Fri, Apr 7, 2017 at 2:16 PM, ytx <[hidden email]> wrote:

> I just want to know how to load the dictionary file when loading data.
> example:load data inpath '/person.txt' into table person
> OPTIONS('DELIMITER'='|','QUOTECHAR'='"','FILEHEADER'='
> id,name,birthday,start_time,work_locations,course','
> COMPLEX_DELIMITER_LEVEL_1'='$','ALL_DICTIONARY_PATH'='/opt/
> dictionary/allcolumn.dictionary');
> When I add content in the allcolumn.dictionary file, the load fails, but
> the
> content is empty and loaded successfully. So I wonder if it's my
> allcolumn.dictionary is wrong? AND if it is added ALL_DICTIONARY_PATH what
> effect?
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/how-to-load-the-
> dictionary-file-when-loading-data-tp10459.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>