crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Re: Saving collection to text files in Scrunch
Date Thu, 29 Nov 2012 16:03:56 GMT
Hey Roman,

While I take a look at that, would you try using the writeTextFile function
(e.g., writeTextFile(<pcollection>, args(1)) ) and let me know if that does
the trick?


On Thu, Nov 29, 2012 at 6:34 AM, Roman V. Shapovalov <> wrote:

> Dear crunch-users,
> I am trying to solve some toy MapReduce problem using Scrunch. When I
> write the final result in the pipeline app, i.e. call
> write(to.textFile(args(1)))
> and get object names in the output file, like:
> org.apache.avro.mapred.AvroWrapper@80
> org.apache.avro.mapred.AvroWrapper@17a73
> This happens only if I perform some mapping (even identity); just
> reading and writing results in good strings in the file.
> It seems that mapping wraps the strings using the AvroWrapper, but
> writing to the text file does not unwrap them. Is it supposed to
> unwrap them?
> There is a factory method To.formattedFile() in Crunch (I guess it may
> help, but it is not documented), but it is not ported to Scrunch. Is
> there another idiom for writing strings?
> Thanks in advance,
> Roman

Director of Data Science
Cloudera <>
Twitter: @josh_wills <>

View raw message