sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Robson <David.Rob...@software.dell.com>
Subject RE: Sqoop to Oracle transfer rates
Date Wed, 18 Jun 2014 23:43:20 GMT
Hi Brenden,

I did some benchmarking a while ago on a 62858MB folder. The best I achieved with 24 mappers
was 867 seconds (72.5MB/s). This was on a 4 node Hadoop cluster that was on some old servers
we had so I'm sure with some better hardware you could get a lot faster.

I'll attach some graphs which you might find interesting - basically you can see OraOop reduces
load on the DB a little bit with a few optimizations - but once you use direct path and partition
exchange loading the benefits are massive. Of course this requires you to have the relevant
Oracle licenses so may not be an option.

David

From: Brenden Cobb [mailto:Brenden.Cobb@humedica.com]
Sent: Thursday, 19 June 2014 6:25 AM
To: user@sqoop.apache.org
Subject: Sqoop to Oracle transfer rates

Looking for some benchmarks on Sqoop (or Oraoop) exports.

Appreciate if anyone feels like sharing some metrics. Rough numbers for MB/s would suffice,
perhaps number of mappers used. I'm working specifically with Oracle exports, but other RDBMS
export rates would be enlightening as well.

Thanks

Mime
View raw message