|
using spark sql + carbondata for data load and query, we often found that the time coast of driver is much more high,especially in concurrent query scenarios. some times the total query time coast is 4s but the the driver coast more than 3s. This needs pay more attention.
under driver the communication with hive metastore or namemode is an obvious performance risk.
both hive metastore and namemode can be accessing slowly, do we create some methods to slove the problem?
|