sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Robson <David.Rob...@software.dell.com>
Subject RE: Final Jiras and Release Date
Date Wed, 23 Jul 2014 06:41:30 GMT
Yes OraOop is not enabled by default - and also this particular issue is only when using partitioning
- so they would have turned on some extra flags. Then they also have to be using 1 more mapper
than requested as you said - so it's a pretty specific combination of events - so I don't
think we need to hold up the release for it.

Do you recall what the behaviour should be in regards to this - the documentation leads me
to believe if we request 4 mappers we should get 4 mappers? If this is only a request and
we may get more - perhaps we should update the documentation about this. Otherwise we could
change the code to guarantee no more than 4 mappers?

-----Original Message-----
From: Abraham Elmahrek [mailto:abe@cloudera.com] 
Sent: Wednesday, 23 July 2014 3:16 PM
To: dev@sqoop.apache.org
Subject: Re: Final Jiras and Release Date

Hey guys,

AFAIK we can fall back onto the original connector? With that being said, OraOop is an awesome
connector... So i'm completely open to waiting for a fix if it isn't too large.

David, I'm assuming you're referring to the "-m" option? I do recall some cases where there
may be one more split than desired if splitting is not clean. Hopefully Venkat has more insight!

-Abe


On Tue, Jul 22, 2014 at 9:42 PM, David Robson < David.Robson@software.dell.com> wrote:

> Hey Venkat,
>
> I had a look into that issue - it seems the problem is even though you 
> request 4 mappers - Sqoop runs 5 mappers. Is the number of mappers 
> meant to be guaranteed? For example if I say 4 mappers, should I get 4 
> mappers? I guess there could be potential to get less mappers if the 
> data was for example 1 row - then that would only be processed by 1 
> mapper. But should it be possible to get more mappers than requested?
>
> OraOop is assuming there will be no more mappers than requested - so 
> if this is a valid scenario we would need to modify OraOop. On the 
> other hand if this is unexpected behaviour then we should look at why 
> the num mappers is not working?
>
> Either way I don't think we need to fix it for this release - as it 
> seems customers would be unlikely to hit this issue.
>
> David
>
> -----Original Message-----
> From: Venkat Ranganathan [mailto:vranganathan@hortonworks.com]
> Sent: Wednesday, 23 July 2014 1:04 PM
> To: dev@sqoop.apache.org
> Subject: Re: Final Jiras and Release Date
>
> Abe
>
> Do we want to target SQOOP-1388 also for 1.4.5 - the one that Vidya 
> has raised for one of the Oracle connector export failure?
>
> My avro patch is in RB.
>
> Thanks
>
> Venkat
>
> On Tue, Jul 22, 2014 at 6:12 PM, Abraham Elmahrek <abe@cloudera.com>
> wrote:
> > Hey guys,
> >
> > It looks like we still have a couple of Jiras still open. Let's see 
> > how things look tomorrow, but let's have these be the last two Jiras 
> > slotted for this release.
> >
> > -Abe
> >
> >
> > On Fri, Jul 18, 2014 at 1:20 PM, Venkat Ranganathan < 
> > vranganathan@hortonworks.com> wrote:
> >
> >> Sounds good.   I have already uploaded the patch for SQOOP-1358.   It
> >> needs to be reviewed by a committer and then committed if no 
> >> further changes are needed (or more work needed otherwise).
> >>
> >> Thanks for driving the release!
> >>
> >> Venkat
> >>
> >> On Fri, Jul 18, 2014 at 10:58 AM, Jarek Jarcec Cecho 
> >> <jarcec@apache.org>
> >> wrote:
> >> > Seems reasonable to me, thank you for driving the release Abe!
> >> >
> >> > Jarcec
> >> >
> >> > On Jul 18, 2014, at 10:40 AM, Abraham Elmahrek <abe@cloudera.com>
> wrote:
> >> >
> >> >> Hey guys,
> >> >>
> >> >> I haven't seen a -1 on this suggestion. So let's create a 
> >> >> release branch come Wednesday July 23rd.
> >> >>
> >> >> -Abe
> >> >>
> >> >>
> >> >> On Wed, Jul 16, 2014 at 11:38 AM, Abraham Elmahrek 
> >> >> <abe@cloudera.com>
> >> wrote:
> >> >>
> >> >>> Are there any objections to this branch date?
> >> >>>
> >> >>> -Abe
> >> >>>
> >> >>>
> >> >>> On Mon, Jul 14, 2014 at 1:22 PM, Abraham Elmahrek 
> >> >>> <abe@apache.org>
> >> wrote:
> >> >>>
> >> >>>> Hey guys,
> >> >>>>
> >> >>>> It looks like there are a couple of Jiras that are in progress

> >> >>>> and slotted for the 1.4.5 release. If we aim for July 23rd
to 
> >> >>>> create a
> >> release
> >> >>>> branch, does that give every one enough time to finish up what

> >> >>>> they're working on?
> >> >>>>
> >> >>>> See https://issues.apache.org/jira/browse/SQOOP-1353 for more

> >> >>>> information.
> >> >>>>
> >> >>>> -Abe
> >> >>>>
> >> >>>
> >> >>>
> >> >
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or 
> >> entity to which it is addressed and may contain information that is 
> >> confidential, privileged and exempt from disclosure under 
> >> applicable law. If the reader of this message is not the intended 
> >> recipient, you are hereby notified that any printing, copying, 
> >> dissemination, distribution, disclosure or forwarding of this 
> >> communication is strictly prohibited. If you have received this 
> >> communication in error, please contact the sender immediately and 
> >> delete it from your
> system. Thank You.
> >>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or 
> entity to which it is addressed and may contain information that is 
> confidential, privileged and exempt from disclosure under applicable 
> law. If the reader of this message is not the intended recipient, you 
> are hereby notified that any printing, copying, dissemination, 
> distribution, disclosure or forwarding of this communication is 
> strictly prohibited. If you have received this communication in error, 
> please contact the sender immediately and delete it from your system. Thank You.
>
Mime
View raw message