[GitHub] incubator-carbondata pull request #656: [CARBONDATA-773] Fixed multiple Dict...

[qiuchenjian-2]

Github user gvramana commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/656#discussion_r106596051
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchema.scala ---
    @@ -524,8 +524,9 @@ case class LoadTable(
                 @throws[Exception]
                 def call: DictionaryServer = {
                   Thread.currentThread().setName("Dictionary server")
    -              val server: DictionaryServer = new DictionaryServer
    -              server.startServer(dictionaryServerPort.toInt)
    +              val server: DictionaryServer = DictionaryServer
    --- End diff --
   
    Dataloading shouldnot continue if dictionary server is failure.
    port initialization should happen and then port should be passed to LoadRDD


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user gvramana commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/656#discussion_r106593403
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/dictionary/server/DictionaryServer.java ---
    @@ -43,42 +43,71 @@
     
       private EventLoopGroup boss;
       private EventLoopGroup worker;
    +  private int port;
    +  private static DictionaryServer INSTANCE = null;
    +
    +  private DictionaryServer(int port) {
    +    startServer(port);
    +  }
    +
    +  public static DictionaryServer getInstance(int port) {
    +    if (INSTANCE == null) {
    --- End diff --
   
    Dictionary server creation should be thread safe, use double check lock .
    if(INSTANCE == null){
      aquirelock
    if(INSTANCE==null){
    createinstance
    }
    releaselock
    }
    return instance


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user gvramana commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/656#discussion_r106594167
 
    --- Diff: integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala ---
    @@ -963,15 +962,14 @@ object CarbonDataRDDFactory {
                           .getTableName
                     }")
                 LOGGER.error("Dataload failed due to failure in table status updation.")
    -            shutDownDictionaryServer(carbonLoadModel, result, false)
                 throw new Exception(errorMessage)
               }
             } else if (!carbonLoadModel.isRetentionRequest) {
               // TODO : Handle it
               LOGGER.info("********Database updated**********")
             }
     
    -        shutDownDictionaryServer(carbonLoadModel, result)
    +        writeDictionary(carbonLoadModel, result)
    --- End diff --
   
    Write dictionary should happen before metadata update, if any failure in writedictionary should fail data load.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user gvramana commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/656#discussion_r106595169
 
    --- Diff: integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala ---
    @@ -963,15 +962,14 @@ object CarbonDataRDDFactory {
                           .getTableName
                     }")
                 LOGGER.error("Dataload failed due to failure in table status updation.")
    -            shutDownDictionaryServer(carbonLoadModel, result, false)
                 throw new Exception(errorMessage)
               }
             } else if (!carbonLoadModel.isRetentionRequest) {
               // TODO : Handle it
               LOGGER.info("********Database updated**********")
             }
     
    -        shutDownDictionaryServer(carbonLoadModel, result)
    +        writeDictionary(carbonLoadModel, result)
    --- End diff --
   
    Create a new WriteTableDictionary method/message to ensure only that only one table dictionary is flushed, while allowing other dictionary generation to continue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---