[jira] [Created] (CARBONDATA-440) Provide Update/Delete functionality support in CarbonData

[Akash R Nilugal (Jira)]

Aniket Adnaik created CARBONDATA-440:
----------------------------------------

             Summary: Provide Update/Delete functionality support in CarbonData
                 Key: CARBONDATA-440
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-440
             Project: CarbonData
          Issue Type: New Feature
          Components: core, data-query, file-format, spark-integration, sql
    Affects Versions: 0.1.1-incubating, 0.1.0-incubating, 0.2.0-incubating
            Reporter: Aniket Adnaik
             Fix For: 0.3.0-incubating


Currently, CarbonData does not support modification of existing rows in the table. This is a major limitation for many real world desirable use cases in data warehousing, such as slow changing dimension tables, data correction of fact tables or data cleanup, etc. Many users want to be able to update and delete rows from the CarbonData table.

Following are some high level design goals to support this functionality,
1. Provide a standard SQL interface to perform Update and Delete operations.
2. Perform Update and Delete operations on CarbonData table without having to rewrite the entire CarbonData block (file) by making use of differential files (a.k.a delta files).
3. After Update or Delete operation, CarbonData readers should skip deleted records and read updated records seamlessly without having to modify user applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[sujith chacko]

Hi Aniket,

I think it will be better if we can consider delete and update compaction
feature also in the high level design list.

Thanks,
Sujith

On Nov 23, 2016 4:29 AM, "Aniket Adnaik (JIRA)" <[hidden email]> wrote:

> Aniket Adnaik created CARBONDATA-440:
> ----------------------------------------
>
>              Summary: Provide Update/Delete functionality support in
> CarbonData
>                  Key: CARBONDATA-440
>                  URL: https://issues.apache.org/jira/browse/CARBONDATA-440
>              Project: CarbonData
>           Issue Type: New Feature
>           Components: core, data-query, file-format, spark-integration, sql
>     Affects Versions: 0.1.1-incubating, 0.1.0-incubating, 0.2.0-incubating
>             Reporter: Aniket Adnaik
>              Fix For: 0.3.0-incubating
>
>
> Currently, CarbonData does not support modification of existing rows in
> the table. This is a major limitation for many real world desirable use
> cases in data warehousing, such as slow changing dimension tables, data
> correction of fact tables or data cleanup, etc. Many users want to be able
> to update and delete rows from the CarbonData table.
>
> Following are some high level design goals to support this functionality,
> 1. Provide a standard SQL interface to perform Update and Delete
> operations.
> 2. Perform Update and Delete operations on CarbonData table without having
> to rewrite the entire CarbonData block (file) by making use of differential
> files (a.k.a delta files).
> 3. After Update or Delete operation, CarbonData readers should skip
> deleted records and read updated records seamlessly without having to
> modify user applications.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>