[GitHub] carbondata pull request #2656: [ExternalFormat] block some operations on ext...

[qiuchenjian-2]

GitHub user Sssan520 opened a pull request:

    https://github.com/apache/carbondata/pull/2656

    [ExternalFormat] block some operations on external format table

    These operations will be blocked:
    1. update data on external format table
    2. delete data on external format table
    3. add columns on external format table
    4. drop columns on external format table
    5. change column datatype on external format table
    6. create external format table if table columns not in format.header
    And add external format information for command 'desc formatted table',
    e.g.
    CREATE TABLE fact_carbon_csv_table(empname String, empno int, designation string)
     STORED BY 'carbondata'
     TBLPROPERTIES(
     'format'='csv',
     'csv.header'='empno, empname, designation, salary',
     'csv.delimiter'='|'
    )
    desc formatted fact_carbon_csv_table:
    +--------------------------------------+--------------------------------------------+
    |col_name                              |data_type                                   |
    +--------------------------------------+--------------------------------------------+
    |empname                               |string                                      |
    |empno                                 |int                                         |
    |designation                           |string                                      |
    |                                      |                                            |
    |##Detailed Table Information          |                                            |
    |Database Name                         |default                                     |
    |Table Name                            |fact_carbon_csv_table                       |
    |CARBON Store Path                     |D:/projects/IdeaProjects/carbondata/........|
    |Comment                               |                                            |
    |Table Block Size                      |1024 MB                                     |
    |Table Data Size                       |0                                           |
    |Table Index Size                      |0                                           |
    |Last Update Time                      |0                                           |
    |SORT_SCOPE                            |LOCAL_SORT                                  |
    |CACHE_LEVEL                           |BLOCK                                       |
    |Streaming                             |false                                       |
    |Local Dictionary Enabled              |false                                       |
    |                                      |                                            |
    |##Detailed External-Format Information|                                            |
    |format                                |csv                                         |
    |csv.delimiter                         ||                                           |
    |csv.header                            |empno,empname,designation,salary            |
    |                                      |                                            |
    |##Detailed Column property            |                                            |
    |ADAPTIVE                              |                                            |
    |SORT_COLUMNS                          |empname,designation                         |
    +--------------------------------------+--------------------------------------------+
    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [x] Any interfaces changed?
           No
     - [x] Any backward compatibility impacted?
           No
     - [x] Document update required?
           No
     - [x] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           has added test case.
     - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
           NA


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/Sssan520/carbondata external-format-ddl

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2656.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2656
   
----
commit 3b501f9bf5aa4319084a857f4f955b3c017a27ac
Author: Sssan520 <liangap2008@...>
Date:   2018-07-02T11:12:24Z

    [external-format] block some ddl,dml operations on external table

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6739/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8016/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6365/



---

[qiuchenjian-2]

Github user Sssan520 commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6741/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8018/



---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212587904
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/mutation/CarbonProjectForDeleteCommand.scala ---
    @@ -49,7 +49,9 @@ private[sql] case class CarbonProjectForDeleteCommand(
         if (!carbonTable.getTableInfo.isTransactionalTable) {
           throw new MalformedCarbonCommandException("Unsupported operation on non transactional table")
         }
    -
    +    if (carbonTable.isExternalFormat) {
    +      throw new MalformedCarbonCommandException("Unsupported operation on external table")
    --- End diff --
   
    'on external format table', not 'on external table'


---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212587770
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/externalformat/CsvBasedCarbonTableSuite.scala ---
    @@ -241,4 +243,170 @@ class CsvBasedCarbonTableSuite extends QueryTest
         CarbonProperties.getInstance().addProperty(CarbonCommonConstants.ENABLE_VECTOR_READER,
           CarbonCommonConstants.ENABLE_VECTOR_READER_DEFAULT)
       }
    +
    +  test("block update data on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    // block update data on external table.
    +    var exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"update $csvCarbonTable set (empname) = ('b') where designation = 'a' ").collect
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block delete data on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"delete from $csvCarbonTable where designation = 'a' ")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block add columns on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable add columns(addcolumn int)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block drop columns on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable drop columns(empname,empno)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block change datatype on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable change empno empno bigint")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block create external format table if columns not in format.header") {
    +    val exception = intercept[ProcessMetaDataException] {
    +      sql(
    +        s"""
    +           | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +           | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +           | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +           | attendance String, utilization String,salary String)
    +           | STORED BY 'carbondata'
    +           | TBLPROPERTIES(
    +           | 'format'='csv',
    +           | 'csv.header'='empname, designation, doj,
    +           | workgroupcategory, workgroupcategoryname, deptno,
    +           | deptname, projectcode, projectjoindate, projectenddate,
    +           | attendance, utilization, SALARY',
    +           | 'csv.delimiter'='|'
    +           | )
    +       """.stripMargin
    +      )
    +    }
    +    assert(exception.getMessage.contains("column 'empno' not in tblproperty 'csv.header'"))
    +  }
    +
    +  test("test desc formatted for external format table") {
    +    sql(
    --- End diff --
   
    better to extract the createTableCommand to a function to reduce duplicated code


---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212588064
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java ---
    @@ -1276,4 +1278,13 @@ public static boolean hasMVDataMap(CarbonTable carbonTable) throws IOException {
         }
         return false;
       }
    +
    +  /**
    +   * if format is 'carbondata' or '' or null, return false
    +   * else return true.
    +   */
    +  public boolean isExternalFormat() {
    --- End diff --
   
    please optimize it to 'isExternalFormatTable'


---

[qiuchenjian-2]

Github user Sssan520 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212604024
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/mutation/CarbonProjectForDeleteCommand.scala ---
    @@ -49,7 +49,9 @@ private[sql] case class CarbonProjectForDeleteCommand(
         if (!carbonTable.getTableInfo.isTransactionalTable) {
           throw new MalformedCarbonCommandException("Unsupported operation on non transactional table")
         }
    -
    +    if (carbonTable.isExternalFormat) {
    +      throw new MalformedCarbonCommandException("Unsupported operation on external table")
    --- End diff --
   
    fixed


---

[qiuchenjian-2]

Github user Sssan520 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212604056
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/externalformat/CsvBasedCarbonTableSuite.scala ---
    @@ -241,4 +243,170 @@ class CsvBasedCarbonTableSuite extends QueryTest
         CarbonProperties.getInstance().addProperty(CarbonCommonConstants.ENABLE_VECTOR_READER,
           CarbonCommonConstants.ENABLE_VECTOR_READER_DEFAULT)
       }
    +
    +  test("block update data on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    // block update data on external table.
    +    var exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"update $csvCarbonTable set (empname) = ('b') where designation = 'a' ").collect
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block delete data on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"delete from $csvCarbonTable where designation = 'a' ")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block add columns on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable add columns(addcolumn int)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block drop columns on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable drop columns(empname,empno)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block change datatype on external format table") {
    +    sql(
    +      s"""
    +         | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +         | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +         | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +         | attendance String, utilization String,salary String)
    +         | STORED BY 'carbondata'
    +         | TBLPROPERTIES(
    +         | 'format'='csv',
    +         | 'csv.header'='eMPno, empname,designation, doj, workgroupcategory,
    +         | workgroupcategoryname, deptno, deptname, projectcode, projectjoindate, projectenddate,
    +         |  attendance, utilization, SALARY'
    +         | )
    +       """.stripMargin
    +    )
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable change empno empno bigint")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external table"))
    +  }
    +
    +  test("block create external format table if columns not in format.header") {
    +    val exception = intercept[ProcessMetaDataException] {
    +      sql(
    +        s"""
    +           | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +           | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +           | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +           | attendance String, utilization String,salary String)
    +           | STORED BY 'carbondata'
    +           | TBLPROPERTIES(
    +           | 'format'='csv',
    +           | 'csv.header'='empname, designation, doj,
    +           | workgroupcategory, workgroupcategoryname, deptno,
    +           | deptname, projectcode, projectjoindate, projectenddate,
    +           | attendance, utilization, SALARY',
    +           | 'csv.delimiter'='|'
    +           | )
    +       """.stripMargin
    +      )
    +    }
    +    assert(exception.getMessage.contains("column 'empno' not in tblproperty 'csv.header'"))
    +  }
    +
    +  test("test desc formatted for external format table") {
    +    sql(
    --- End diff --
   
    fixed


---

[qiuchenjian-2]

Github user Sssan520 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212604083
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java ---
    @@ -1276,4 +1278,13 @@ public static boolean hasMVDataMap(CarbonTable carbonTable) throws IOException {
         }
         return false;
       }
    +
    +  /**
    +   * if format is 'carbondata' or '' or null, return false
    +   * else return true.
    +   */
    +  public boolean isExternalFormat() {
    --- End diff --
   
    fixed


---

[qiuchenjian-2]

Github user Sssan520 commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    retest this please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6390/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6766/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8044/



---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2656#discussion_r212787386
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/externalformat/CsvBasedCarbonTableSuite.scala ---
    @@ -241,4 +243,104 @@ class CsvBasedCarbonTableSuite extends QueryTest
         CarbonProperties.getInstance().addProperty(CarbonCommonConstants.ENABLE_VECTOR_READER,
           CarbonCommonConstants.ENABLE_VECTOR_READER_DEFAULT)
       }
    +
    +  test("block update data on external format table") {
    +    createCsvCaronTable
    +    // block update data on external table.
    +    var exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"update $csvCarbonTable set (empname) = ('b') where designation = 'a' ").collect
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external format table"))
    +  }
    +
    +  test("block delete data on external format table") {
    +    createCsvCaronTable
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"delete from $csvCarbonTable where designation = 'a' ")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external format table"))
    +  }
    +
    +  test("block add columns on external format table") {
    +    createCsvCaronTable
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable add columns(addcolumn int)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external format table"))
    +  }
    +
    +  test("block drop columns on external format table") {
    +    createCsvCaronTable
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable drop columns(empname,empno)")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external format table"))
    +  }
    +
    +  test("block change datatype on external format table") {
    +    createCsvCaronTable
    +    val exception = intercept[MalformedCarbonCommandException] {
    +      sql(s"alter table $csvCarbonTable change empno empno bigint")
    +    }
    +    assert(exception.getMessage.contains("Unsupported operation on external format table"))
    +  }
    +
    +  test("block create external format table if columns not in format.header") {
    +    val exception = intercept[ProcessMetaDataException] {
    +      sql(
    +        s"""
    +           | CREATE TABLE $csvCarbonTable(empname String, empno smallint, designation string,
    +           | deptname String, projectcode int, projectjoindate String,projectenddate String,
    +           | doj String, workgroupcategory int, workgroupcategoryname String,deptno int,
    +           | attendance String, utilization String,salary String)
    +           | STORED BY 'carbondata'
    +           | TBLPROPERTIES(
    +           | 'format'='csv',
    +           | 'csv.header'='empname, designation, doj,
    +           | workgroupcategory, workgroupcategoryname, deptno,
    +           | deptname, projectcode, projectjoindate, projectenddate,
    +           | attendance, utilization, SALARY',
    +           | 'csv.delimiter'='|'
    +           | )
    +       """.stripMargin
    +      )
    +    }
    +    assert(exception.getMessage.contains("column 'empno' not in tblproperty 'csv.header'"))
    +  }
    +
    +  test("test desc formatted for external format table") {
    +    createCsvCaronTable
    +    val result = sql(s"desc formatted $csvCarbonTable").collect()
    +    val index = result.indexWhere(_.get(0).toString.
    +      contains("##Detailed External-Format Information"))
    +    assert(index > -1)
    +    assert(result(index + 1).get(0).toString.contains("format"))
    --- End diff --
   
    the sequence of the content is not ensured, it may change in the future.
    You can use checkExistence to check whether the keywords are there in the DataFrame


---

[qiuchenjian-2]

Github user xuchuanyin commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    In addition, I think you can rename the test suite to 'CsvExternalFormatBasicFunctionSuite'


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2656
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8061/



---