[GitHub] carbondata pull request #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION...

[qiuchenjian-2]

GitHub user dhatchayani opened a pull request:

    https://github.com/apache/carbondata/pull/1574

    [CARBONDATA-1816] Changing BAD_RECORDS_ACTION default action to FAIL

    Currently, the default action is FORCE, this may allow the user by default to load bad records also. So, changing the default action of BAD_RECORDS_ACTION to FAIL will alert the user if any bad_records are there in the loading data.
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [ ] Testing done
           
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhatchayani/incubator-carbondata bad_rec_action

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1574.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1574
   
----
commit 40c2c3d111e8b100cbf42e38c60649caed461d46
Author: dhatchayani <[hidden email]>
Date:   2017-11-27T09:45:17Z

    [CARBONDATA-1816] Changing BAD_RECORDS_ACTION default action to FAIL

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1488/



---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1490/



---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1492/



---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    retest sdv please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1499/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1503/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1506/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1509/



---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1919/



---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1574#discussion_r153420051
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
    @@ -1276,7 +1276,7 @@
       @CarbonProperty
       public static final String CARBON_BAD_RECORDS_ACTION = "carbon.bad.records.action";
     
    -  public static final String CARBON_BAD_RECORDS_ACTION_DEFAULT = "FORCE";
    +  public static final String CARBON_BAD_RECORDS_ACTION_DEFAULT = "FAIL";
    --- End diff --
   
    Can you add comment to describe the behavior for 'FAIL'


---

[qiuchenjian-2]

Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1574#discussion_r153428387
 
    --- Diff: integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala ---
    @@ -1067,7 +1067,7 @@ class AlterTableTestCase extends QueryTest with BeforeAndAfterAll {
       val p3 = prop.getProperty("carbon.horizontal.delete.compaction.threshold", CarbonCommonConstants.DEFAULT_DELETE_DELTAFILE_COUNT_THRESHOLD_IUD_COMPACTION)
       val p4 = prop.getProperty("carbon.compaction.level.threshold", CarbonCommonConstants.DEFAULT_SEGMENT_LEVEL_THRESHOLD)
       val p5 = prop.getProperty("carbon.enable.auto.load.merge", CarbonCommonConstants.DEFAULT_ENABLE_AUTO_LOAD_MERGE)
    -  val p6 = prop.getProperty("carbon.bad.records.action", CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION_DEFAULT)
    +  val p6 = prop.getProperty("carbon.bad.records.action", "FORCE")
    --- End diff --
   
    instead of hard coding "FORCE" use enum LoggerAction.Force.name
    Do it everywhere you have made the code changes


---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1952/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1554/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1953/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1574
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1954/



---