sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joshua Baxter <joshuagbax...@gmail.com>
Subject Re: Using more than a single mapper per partition with OraOop
Date Mon, 01 Dec 2014 13:38:04 GMT
Hi David,

Sorry for the slow reply,  I haven't had access to the cluster for the last
couple of weeks. This patch seems to have fixed the issue and i am now able
to pull down evenly sized parquet files with Oraoop.

On the issue of the time taken by the block fetching. I forced a higher DOP
for the session to make it faster and used the noparallel hint to stop the
mappers from using additional resource. Though i could see the settings had
taken effect from the query monitor, on repeated runs it didn't really take
much off the initial query time. If anything it made it slightly longer.
But the additional overhead for us isn't so important as we won't usually
be dumping partitions more frequently than once a day.

Thanks very much for your help.

Josh


On Wed, Nov 19, 2014 at 3:13 AM, David Robson <
David.Robson@software.dell.com> wrote:

>  Hi Joshua,
>
>
>
> I have submitted a patch to change the int to a long – would you be able
> to test this and see if it resolves your issue?
>
>
>
> https://issues.apache.org/jira/browse/SQOOP-1764
>
>
>
> Are you able to compile it – or I can send you a compiled version if you
> like?
>
>
>
> David
>
>
>
> *From:* Joshua Baxter [mailto:joshuagbaxter@gmail.com]
> *Sent:* Tuesday, 11 November 2014 2:53 AM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Using more than a single mapper per partition with OraOop
>
>
>
> Thanks for your help David. Can you let us know the issue number once it
> is logged.
>
>
>
> Cheers
>
>
>
> Josh
>

Mime
View raw message