[Discussion] Presto read support for complex data types

[akshay_nuthala]

*Background*: This feature will enable Presto to read complex columns from
carbondata file.
Complex columns include - array, map and struct.

NOTE: This design only handles array type. Map and struct data types will be
handled later.

Details of solution and implementation is mentioned in the document attached
in JIRA.
https://issues.apache.org/jira/browse/CARBONDATA-3830

Thanks,
Akshay



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Ajantha Bhat]

+ 1,
This is really required as complex schema is very common now a days and
most the user have it.

I see that current design covers only 1 level array.
Multi level array with complex children and other complex type also need to
be supported.
Now, that you have an idea about array. It is better to design for struct
and Map. So that you can come up with generic recursive methods to avoid
rework later.

Thanks,
Ajantha

On Fri, May 22, 2020 at 1:13 PM akshay_nuthala <[hidden email]>
wrote:

> *Background*: This feature will enable Presto to read complex columns from
> carbondata file.
> Complex columns include - array, map and struct.
>
> NOTE: This design only handles array type. Map and struct data types will
> be
> handled later.
>
> Details of solution and implementation is mentioned in the document
> attached
> in JIRA.
> https://issues.apache.org/jira/browse/CARBONDATA-3830
>
> Thanks,
> Akshay
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[akshay_nuthala]

Design document for handling multi-level array has been updated. Please refer
to the following JIRA sub-task.

https://issues.apache.org/jira/browse/CARBONDATA-3850


Thanks,
Akshay



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[akashrn5]

+1

It's good to support complex for presto. Please include other types of map
and struct also in th same document.

If  you can elaborate more on the implementation of nested array types in
the design doc, it would be helpful.

Regards
Akash

On Wed, Jun 10, 2020, 9:11 PM akshay_nuthala <[hidden email]>
wrote:

> Design document for handling multi-level array has been updated. Please
> refer
> to the following JIRA sub-task.
>
> https://issues.apache.org/jira/browse/CARBONDATA-3850
>
>
> Thanks,
> Akshay
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>