sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Tran <br...@box.com>
Subject Sqoop downloads split into chunks
Date Thu, 24 May 2012 07:04:22 GMT
Hi Sqoop gurus,

I currently use Sqoop to import from MySQL into HDFS.

Some of the tables that I import have become significantly larger to the
point that a full dump significantly slows down the host.

I would like to split the imports into smaller chunks, but limit the number
of chunks I download in parallel to avoid significant load on the server.

Is there anything in Sqoop that provides this functionality?

The closest thing I could find in the Sqoop user guide was the
--num-mappers option, but using it to download in smaller chunks would
increase the server load as all the chunks are downloaded in parallel.

Thanks!

Brian

Mime
View raw message