spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: Streaming files as a whole
Date Thu, 30 Jan 2014 13:55:11 GMT
Hi,
I am using Spark Streaming for this, in Streaming I am trying to open the
file as text file and Dstream.
Regards
Mayur

Mayur Rustagi
Ph: +919632149971
h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
https://twitter.com/mayur_rustagi



On Thu, Jan 30, 2014 at 7:17 PM, Woody Christy <wchristy@cloudera.com>wrote:

> Take a look at the Mahout xmlinputformat class. That should get  you
> started.
>
>
> On Thu, Jan 30, 2014 at 5:08 AM, Mayur Rustagi <mayur.rustagi@gmail.com>wrote:
>
>> I am trying to load xml in streaming and convert to csv and store it.
>> When I use textfile it separates the file on "\n" and hence breaks the
>> parser. Is it possible to receive the data one file at a time from the hdfs
>> folder ?
>>
>> Mayur Rustagi
>> Ph: +919632149971
>> h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
>> https://twitter.com/mayur_rustagi
>>
>
>
>
> --
>
> Woody Christy
> Solutions Architect | Partner Engineering | Cloudera Inc
> @woodychristy
>
>
>
>

Mime
View raw message