How to look up date segment details in carbon without partition.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How to look up date segment details in carbon without partition.

carbondata-newuser
This post was updated on .
It seems carbondata not recommend use partitionby and partitionby is not
supported in global sort scope.
It is very conveniently to look up how many date partition(along with the
partition size every day) already exists in hive(save as parquet).
In carbondata I add the date column to first sort columns in order to using
global sort scope.
But how can I look up segment corresponding date and size in carbondata.
 



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: How to look up date segment details in carbon without partition.

Liang Chen
Administrator
Hi

In Carbondata system, the segment concept may be different with other
system.
One data load is one segment for carbondata.

Actually, carbondata currently support partition with global sort also, you
can use date as partition column to check data size for under each partition
folder.

Regards
Liang

carbondata-newuser wrote

> It seems carbondata not recommend use partitionby and partitionby is not
> supported in global sort scope.
> It is very conveniently to look up how many date partition(along with the
> partition size every day) already exists in hive(save as parquet).
> In carbondata I add the date column to first sort columns in order to
> using
> global sort scope.
> But how can I look up segment corresponding date and size in carbondata.
>  
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/