sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: Should compression with Avro import work?
Date Wed, 18 Jan 2012 16:54:01 GMT
Hi Lars,

I don't think compression is wired up for Avro imports - would you
mind filing a JIRA for it?

Thanks,
Tom

On Wed, Jan 18, 2012 at 7:16 AM, Lars Francke <lars.francke@gmail.com> wrote:
> Hi,
>
> I've tried using sqoop import with Avro and it works beautifully. But
> I've tried it with compression which according to the documentation
> should work. It didn't for me so before I go any further: Is this a
> problem in the documentation?
>
> I briefly looked at the AvroOutputFormat and it doesn't do any
> compression as far as I can tell. The job.xml is certainly correct in
> that it mentions mapred.output.compress=true.
>
> Removing the --as-avrodatafile option gives me a compressed text file
> so I'm pretty sure I've got the command line right.
>
> Cheers,
> Lars

Mime
View raw message