[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3566: Added prepriming in the Index Server Documentation

[GitBox]

vikramahuja1001 opened a new pull request #3566: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566
 
 
    ### Why is this PR needed?
   
   
    ### What changes were proposed in this PR?
   
       
    ### Does this PR introduce any user interface change?
    - No
    - Yes. (please explain the change and update document)
   
    ### Is any new testcase added?
    - No
    - Yes
   
       
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#issuecomment-571931913
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1525/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r364559033
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
+will be loaded for any filter query. Unless the query is fired on the table there will be no caching,
+which reduces the performance of the first time query. To improve the performance of the first time
+query, cache can be preprimed in the Index Server. During prepriming the datamaps can be loaded in the
+Index Server during load command before a select query is fired on the table.
+
+The user can enable prepriming by using 'carbon.indexserver.enable.prepriming'.
 
 Review comment:
   Mention about the async call and that the load performance will not be degraded.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r364559276
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
 
 Review comment:
   No need to mention count(*). cache can be loaded by any query

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365046175
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
+will be loaded for any filter query. Unless the query is fired on the table there will be no caching,
+which reduces the performance of the first time query. To improve the performance of the first time
+query, cache can be preprimed in the Index Server. During prepriming the datamaps can be loaded in the
+Index Server during load command before a select query is fired on the table.
 
 Review comment:
   during Load command => i think should be for both Load and Insert into command. Please change

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365086674
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
+will be loaded for any filter query. Unless the query is fired on the table there will be no caching,
+which reduces the performance of the first time query. To improve the performance of the first time
+query, cache can be preprimed in the Index Server. During prepriming the datamaps can be loaded in the
+Index Server during load command before a select query is fired on the table.
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365086686
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365086695
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,16 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 table path would be used to write the files.
 
+## Prepriming
+The caching of the datamaps in the Index Server will start once the query is fired on the table for
+the first time. All the datamaps will be loaded if a count(*) query is fired and only the required
+will be loaded for any filter query. Unless the query is fired on the table there will be no caching,
+which reduces the performance of the first time query. To improve the performance of the first time
+query, cache can be preprimed in the Index Server. During prepriming the datamaps can be loaded in the
+Index Server during load command before a select query is fired on the table.
+
+The user can enable prepriming by using 'carbon.indexserver.enable.prepriming'.
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#issuecomment-572912915
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1580/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

chetandb commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365120979
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,20 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 
 Review comment:
   "location for these file" to be changed to "location for these files"

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#discussion_r365127281
 
 

 ##########
 File path: docs/index-server.md
 ##########
 @@ -119,6 +119,20 @@ be written to file.
 The user can set the location for these file by using 'carbon.indexserver.temp.path'. By default
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#issuecomment-572975332
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1581/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on issue #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566#issuecomment-574987460
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3566: [CARBONDATA-3492]: Added prepriming in the Index Server Documentation
URL: https://github.com/apache/carbondata/pull/3566
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services