Re: [DISCUSSION]Support for Geospatial indexing

Posted by Ajantha Bhat on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Support-for-Geospatial-indexing-tp85268p85513.html

Hi Jacky,

we have checked about geomesa 



a. Geomesa is tightly coupled with  key-value pair databases like Accumulo, HBase, Google Bigtable and Cassandra databases and used for OLTP queries.
b. Geomesa current spark integration is only in query flow, load from spark is not supported. spark can be used for analytics on geomesa store. 
Here they override spark catalyst optimizer code to intercept filter from logical relation and they push down to geomesa server. 
All the query logic like spatial time curve building (z curve, quadtree) doesn't happen at spark layer. It happens in geoserver layer which is coupled with key-value pair databases.
https://www.geomesa.org/documentation/user/architecture.html

https://www.geomesa.org/documentation/user/spark/architecture.html

https://www.youtube.com/watch?v=Otf2jwdNaUY

c. Geomesa is for spatio-temporal data , not just a spatial data.
so, we cannot integrate carbon with  geo mesa directly, but we can reuse some of the logic present in it like quadtree formation and look up.

Also I found another alternative "GeoSpark", this project is not coupled with any store. 
https://datasystemslab.github.io/GeoSpark/

https://www.public.asu.edu/~jiayu2/presentation/jia-icde19-tutorial.pdf
so, we will check further about integrating carbon to GeoSpark or reusing some of the code from this.

Also regarding the second point, yes, we can have carbon implementation as a generic framework where we can plugin the different logic. 

Thanks,
Ajantha





On Mon, Oct 21, 2019 at 6:34 PM Indhumathi <[hidden email]> wrote:
Hi Venu,

I have some questions regarding this feature.

1. Does geospatial index supports on streaming table?. If so, will there be
any impact on generating
    geoIndex on steaming data?
2. Does it have any restrictions on sort_scope?
3. Apart from Point and Polygon queries, will geospatial index also support
Aggregation queries on
    geographical location data?

Thanks & Regards,
Indhumathi




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/