spark streaming insert data error

Posted by Jocean shi on Mar 20, 2019; 7:16am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/spark-streaming-insert-data-error-tp76128.html

Hi:

I use spark streaming insert data into carbondata. I find that the schema
of my data don't match the schema of table.
after the method of class  RowParserImpl:

@Override
public Object[] parseRow(Object[] row) {
  if (row == null) {
    return new String[numberOfColumns];
  }
  // If number of columns are less in a row then create new array with
same size of header.
  if (row.length < numberOfColumns) {
    String[] temp = new String[numberOfColumns];
    System.arraycopy(row, 0, temp, 0, row.length);
    row = temp;
  }
  Object[] out = new Object[genericParsers.length];
  for (int i = 0; i < genericParsers.length; i++) {
    Object obj = row[inputMapping[i]];
    out[outputMapping[i]] = genericParsers[i].parse(obj);
  }
  return out;
}

The schema is "id,name,city,salary".but the table schema is
"name,city,id,salary" in

the class RowConverterImpl .

why? is bug?


Best

Jocean.shi