[Disscussion] Support Streaming Ingest

[David CaiQiang]

Hi All,

    Currently CarbonData is a storage system supported by an indexed
columnar format. It supports batch data loading from HDFS files or data
source. As new applications like IOT emerging, streaming ingests from the
network, Kafka or other real-time data source to CarbonData becomes a common
requirement.

    The document of the JIRA link explains about the streaming ingest usage
and internal design in CarbonData.
    JIRA link: https://issues.apache.org/jira/browse/CARBONDATA-1572

    Let's disscuss Streaming Ingest feature.

    Any thoughts, comments and questions?

Thanks!



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Liang Chen]

Hi

One question:
Why not supports structured streaming to replace spark streaming ?
-----------------------------------------------
In first phase implementation, it should support kafka and spark streaming
integration. More
streaming framework support is preferable in the future.

Regards
Liang

David CaiQiang wrote

> Hi All,
>
>     Currently CarbonData is a storage system supported by an indexed
> columnar format. It supports batch data loading from HDFS files or data
> source. As new applications like IOT emerging, streaming ingests from the
> network, Kafka or other real-time data source to CarbonData becomes a
> common
> requirement.
>
>     The document of the JIRA link explains about the streaming ingest
> usage
> and internal design in CarbonData.
>     JIRA link: https://issues.apache.org/jira/browse/CARBONDATA-1572
>
>     Let's disscuss Streaming Ingest feature.
>
>     Any thoughts, comments and questions?
>
> Thanks!
>
>
>
> -----
> Best Regards
> David Cai
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/