[Discussion]Float and Double compatibility issue with external segments to Carbon

[VenuReddy]

Hi Community!

Need a help!!

Was trying to create a carbon table with float field. I understand that
internally it stores float as double in its schema.
When we try to add an externa segment with float field, add segment/queries
on ext segment would not be successful due to datatype mismatch.


create table carbon_float(c1 float) stored as carbondata
desc formatted carbon_float

// hive table shows datatype as double.
desc carbon_float
// create another table with same schema
create table orc_float4(c1 float) using orc
insert into orc_float4 select 1

// Add external segment fails
alter table carbon_float4 add segment options('path'='xxx', 'format'='orc')
</raw

Trying to understand the following things:
1. Any reason to make float to double the carbon ? Why not support both ?
2. Why do we change float to double and store in hive table catalog ?


Any help would be much appreciated.




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[David CaiQiang]

It is a historical legacy issue and easy to reuse the solution of the double
data type.

Suggest implementing the float data type independently.




-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Jacky Li]

Hi,

Yes, I think we should correct it.
In the schema, it should be float for float type. In internal store, it is usinng adaptive encoding, so I think it is ok anyway.

Regards,
Jacky

> 2020年5月8日 上午10:23，David CaiQiang <[hidden email]> 写道：
>
> It is a historical legacy issue and easy to reuse the solution of the double
> data type.
>
> Suggest implementing the float data type independently.
>
>
>
>
> -----
> Best Regards
> David Cai
> --
> Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>