[Disscuss] The precise of timestamp is limited to millisecond in carbondata, which is incompatiable with DB&Spark&Hive

[haomarch]

The SimpleDateFormat is used to convert string and timestamp. But
SimpleDateFormat doesn't support nanoseconds and microsecond, which is is
incompatiable with DB&Spark&Hive.

To support nanoseconds and microsecond, we shall change SimpleDateFormat to
DateTimeFormatter.
Shall we involve this work？



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[xubo245]

+1, please consider compatable  for history data



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Ajantha Bhat]

+ 1,
as SimpleDateFormat doesn't support nanoseconds and microsecond.

Thanks,
Ajantha

On Tue, Jul 14, 2020 at 5:03 PM xubo245 <[hidden email]> wrote:

> +1, please consider compatable  for history data
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[Zhangshunyu]

+1



-----
My English name is Sunday
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[ravipesala]

Hi,

I think it is bigger than just changing to DateTimeFormatter. As of now,
carbon uses only 64 bit to store timestamp so it can accommodate till
milliseconds. In order to support till nanoseconds, we need to use 96
bits.  If you check spark-parquet it uses 96 bits to store timestamp. It
would be good if we also go in that direction to support till nanosecond
level. Thank you.

Regards,
Ravindra.

On Wed, 15 Jul 2020 at 14:50, Zhangshunyu <[hidden email]> wrote:

> +1
>
>
>
> -----
> My English name is Sunday
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

--
Thanks & Regards,
Ravi

[kumarvishal09]

Hi,
I agree with Ravi, first we need to come up with solution how to store till
nano/mico second in carbon. Then changing the formatter will be useful for
us.

Regards
Kumar Vishal

On Wed, 15 Jul 2020 at 6:18 PM, Ravindra Pesala <[hidden email]>
wrote:

> Hi,
>
> I think it is bigger than just changing to DateTimeFormatter. As of now,
> carbon uses only 64 bit to store timestamp so it can accommodate till
> milliseconds. In order to support till nanoseconds, we need to use 96
> bits.  If you check spark-parquet it uses 96 bits to store timestamp. It
> would be good if we also go in that direction to support till nanosecond
> level. Thank you.
>
> Regards,
> Ravindra.
>
> On Wed, 15 Jul 2020 at 14:50, Zhangshunyu <[hidden email]> wrote:
>
> > +1
> >
> >
> >
> > -----
> > My English name is Sunday
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> >
>
>
> --
> Thanks & Regards,
> Ravi
>

[David CaiQiang]

agree with Ravi



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/