Login  Register

Re: column auto mapping when loading data from csv file

Posted by David CaiQiang on Mar 13, 2017; 8:42am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/column-auto-mapping-when-loading-data-from-csv-file-tp8717p8753.html

Hi Ravindra,
    How about to use 'NOT_AUTOFILEHEADER'='true' as following?
   I think 'AUTOFILEHEADER'='true' should be a default behavior.

   if (load sql contain "FILEHEADER") {
     1. input files shouldn't contain a fileheader
     2. use "FILEHEADER" parameter to load data after passing column check

   } else {

     if (not exists 'NOT_AUTOFILEHEADER' option) {

       1.auto map the first row of input files with table's columns
       if(the first row contain all column names ) {
          2. use first row as the file header to load data
       } else if (the first row contain part of column names) {
          2. stop loading
       } else {
         2. use the origin order of table's columns to load data
       }

     } else {
       1. input files should contain a file header
       2. use first row as the file header to load data after passing column check
    }
  }
Best Regards
David Cai