Login  Register

Re: [DISCUSSION] Support Time Series for MV datamap and autodatamap loading of timeseries datamaps

Posted by akashnilugal@gmail.com on Sep 24, 2019; 9:13am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Support-Time-Series-for-MV-datamap-and-autodatamap-loading-of-timeseries-datamaps-tp84721p84763.html

Hi Manhua,

Thanks for the questions, Please find the comments
1. User can specify the granularity as minute, so we take as 1 unit, (1 minute in this case) and store the data, then during query in UDF he can mention for minute level query, how many minutes data he wants.

2. Since we will be loading data to datamap from fact, there is no restriction on it.

Regards,
Akash

On 2019/09/24 08:00:55, Manhua <[hidden email]> wrote:

> Hi Akash
>
> Can user specific the granularity? Such as 5minutes, 15 minutes
>
> Is there any constraint on timestamp_column's datatype?  Including DATE,  TIMESTAMP,  BIGINT(Unix timestamp)
>
> On 2019/09/23 13:42:48, Akash Nilugal <[hidden email]> wrote:
> > Hi Community,
> >
> > Timeseries data are simply measurements or events that are
> > tracked,monitored, downsampled, and aggregated over time.
> > Basicallytimeseries data analysis helps in analyzing or monitoring
> > theaggregated data over period of time to take better decision forbusiness.
> > So since carbondata supports olap datamap like preaggregate, MV and since
> > time series is of atmost importance,
> > we can supporttimeseries for carbondata over MV datamap model.
> >
> > Currentlycarbondata supports timeseries on preaggregate datamap, but its
> > analpha feature and there are so many limitations when we compare and
> > analyze the existing timeseries database or projects which supportstime
> > series like apache druid or influxdb. So, in this feature we can support
> > timeseries
> > by avoiding the limitations in the current system. After doing the analysis
> > on the current existing timeseries database like influxdb, and the apache
> > druid,
> > i have  prepared a solution/design document. Any inputs, improvements or
> > suggestion are most welcome.
> >
> > I have created jira https://issues.apache.org/jira/browse/CARBONDATA-3525 for
> > this. Later i will create sub jiras for tracking.
> >
> >
> > Regards,
> > Akash R Nilugal
> >
>