Posted by
Akash R Nilugal (Jira) on
Sep 05, 2016; 11:57am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Commented-CARBONDATA-204-Query-statistics-issue-tp771.html
[
https://issues.apache.org/jira/browse/CARBONDATA-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464840#comment-15464840 ]
ASF GitHub Bot commented on CARBONDATA-204:
-------------------------------------------
Github user foryou2030 commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/123#discussion_r77511283
--- Diff: core/src/main/java/org/apache/carbondata/core/carbon/querystatistics/DriverQueryStatisticsRecorder.java ---
@@ -78,106 +83,148 @@ public synchronized void recordStatisticsForDriver(QueryStatistic statistic, Str
*/
public void logStatisticsAsTableDriver() {
synchronized (lock) {
- String tableInfo = collectDriverStatistics();
- if (null != tableInfo) {
- LOGGER.statistic(tableInfo);
+ Iterator<Map.Entry<String, List<QueryStatistic>>> entries =
+ queryStatisticsMap.entrySet().iterator();
+ while (entries.hasNext()) {
+ Map.Entry<String, List<QueryStatistic>> entry = entries.next();
+ String queryId = entry.getKey();
+ // clear the unknown query statistics
+ if(StringUtils.isEmpty(queryId)) {
+ queryStatisticsMap.remove(queryId);
--- End diff --
ok, handled
> Query statistics issue
> -----------------------
>
> Key: CARBONDATA-204
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-204> Project: CarbonData
> Issue Type: Bug
> Reporter: Gin-zhj
> Assignee: Gin-zhj
>
> Query statistics issue:
> 1. some query statistics that never be printed will be keeped into querystatisticsMap, so it will cause "out of memory" for long time running
> 2. in some sceniaro, the driver can't record "sql_parse_time" , the driver statistics logs will not be output
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)