Re: Improving show segment info

Posted by Ajantha Bhat on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Improving-show-segment-info-tp91874p91907.html

3. And about event time. I don't think we need to keep it for every row. It
is a waste of storage size. can we keep in loadMetadetails or file level ?

On Mon, Feb 17, 2020 at 11:10 AM Ajantha Bhat <[hidden email]> wrote:

> Hi Likun,
>
> I think this display command is hard to maintain if we provide all these
> options manually.
>
> *1. How about creating a "tableName.segmentInfo" child table for each main
> table?* user can query this table and easy to support filter, group by.
> we just have to finalize the schema of this table.
>
> 2. For each partition to find out which all the segments it is mapped to,
> currently we don't store this information anywhere. so, where are you
> planning to store it? I don't think we need to calculate it every time.
>
> Thanks,
> Ajantha
>
>
> On Sun, Feb 16, 2020 at 10:48 PM akashrn5 <[hidden email]> wrote:
>
>> Hi,
>>
>> >I got your point, but given the partition column by user does not help
>> reducing the information. If we want to reduce the >amount of the
>> information, we should ask user to give the filter on partition column
>> like
>> example 3 in my original mail.
>>
>> 1. my concern was if there are more partition column and instead of
>> partition value filter, i was thinking of having filter on partition
>> column
>>
>>
>> >Do you mean skip the partition columns in the SHOW SEGMENTS result? For
>> example 3.
>> Same as above point 1.
>>
>> >No, by showing example 1, actually I want to change the default output of
>> the SHOW SEGMENTS to those 6 fields only >in example 1.
>> >I suggest having both `spent` and `throughput` so that user does not need
>> to calculate himself.
>>
>> I agree, then what about the old default infos which we have now, like
>> data
>> size, index size, mergedTo,format? All these infos will be moved to *DESC
>> SEGMENT 2 ON table1*, if i am not wrong?
>>
>> Regards,
>> Akash R Nilugal
>>
>>
>>
>>
>> --
>> Sent from:
>> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>>
>