Index file cache will not work when the table has invalid segment.

[carbondata-newuser]

Carbon version is 1.4 rc2.
create table(
col1 string,
col2 int,
col2 string,
date string
)

*First step:*
insert into table carbonTest select col1,col2,col3,"20180707" from
hiveTable2 where date="20180707";
The col3 is a hive map type, so this insert will be failed.
And it will create invalid segment. (I'm not sure it is because of this).
*second step:*
insert into carbonTest select col1,col2,"","20180707" from hiveTable2;

Then any query to this table will access the index files time and time
again.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Liang Chen]

Hi

Currently, CarbonData doesn't support map data type

Regards
Liang


carbondata-newuser wrote

> Carbon version is 1.4 rc2.
> create table(
> col1 string,
> col2 int,
> col2 string,
> date string
> )
>
> *First step:*
> insert into table carbonTest select col1,col2,col3,"20180707" from
> hiveTable2 where date="20180707";
> The col3 is a hive map type, so this insert will be failed.
> And it will create invalid segment. (I'm not sure it is because of this).
> *second step:*
> insert into carbonTest select col1,col2,"","20180707" from hiveTable2;
>
> Then any query to this table will access the index files time and time
> again.
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[xuchuanyin]

Hi, liang, I think it may be a problem. The segment with LOAD_FAILED should
not affect the query on the normal segment.

In the previous mail, the second data loading is successful and query on
this segment should use the index file cache.

Besides, if the dataloading is failed, will the failed segment be cleaned?



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/