[
https://issues.apache.org/jira/browse/CARBONDATA-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501865#comment-15501865 ]
ASF GitHub Bot commented on CARBONDATA-257:
-------------------------------------------
GitHub user jackylk opened a pull request:
https://github.com/apache/incubator-carbondata/pull/174 [CARBONDATA-257] Make CarbonData readable through Spark/MapReduce program
User should be able to use SparkContext.newAPIHadoopFile to read CarbonData files
For example:
```
val input = sc.newAPIHadoopFile(s"${cc.storePath}/default/carbon1",
classOf[CarbonInputFormat[Array[Object]]],
classOf[Void],
classOf[Array[Object]])
val result = input.map(x => x._2.toList).collect
result.foreach(x => println(x.mkString(", ")))
```
In this PR, the INPUT_DIR in CarbonInputFormat job configuration is changed to table path instead of store path, since sc.newAPIHadoopFile will set it to the first input parameter (`path` indicating the table path)
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/jackylk/incubator-carbondata inputformat
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/174.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #174
----
commit c67e5a91fc32484dea49ede62dbcea68ba11e98f
Author: jackylk <
[hidden email]>
Date: 2016-09-18T23:47:55Z
change INPUT_DIR to tablePath instead of storePath
commit f2858e247a9e968da3cd4121c31cc6f95456d804
Author: jackylk <
[hidden email]>
Date: 2016-09-18T23:48:35Z
add mapreduce example to read carbon files
----
> Make CarbonData readable through Spark/MapReduce program
> --------------------------------------------------------
>
> Key: CARBONDATA-257
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-257> Project: CarbonData
> Issue Type: Improvement
> Components: hadoop-integration
> Affects Versions: 0.1.0-incubating
> Reporter: Jacky Li
> Assignee: Jacky Li
> Fix For: 0.2.0-incubating
>
>
> User should be able to use SparkContext.newAPIHadoopFile to read CarbonData files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
