sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Final Jiras and Release Date
Date Wed, 23 Jul 2014 21:03:44 GMT
Release branch is alive here:

https://github.com/apache/sqoop/tree/branch-1.4.5

Trunk has been switched to version 1.4.6-SNAPSHOT.

Jarcec

On Jul 23, 2014, at 2:00 PM, Jarek Jarcec Cecho <jarcec@apache.org> wrote:

> Too late, I’m already in process of branching :-) But that’s fine - we can always
cherry-pick whatever will be necessary.
> 
> Jarcec
> 
> On Jul 23, 2014, at 1:57 PM, Abraham Elmahrek <abe@cloudera.com> wrote:
> 
>> Ah actually, it seems there is a broken OraOop test case. Let me see what's
>> going on there before we branch. Jarcec, will you be free around 4PM to
>> branch?
>> 
>> -Abe
>> 
>> 
>> On Wed, Jul 23, 2014 at 1:19 PM, Abraham Elmahrek <abe@cloudera.com> wrote:
>> 
>>> Awesome! Thanks for your hard work guys!
>>> 
>>> Jarcec, can we use
>>> https://github.com/apache/sqoop/commit/81624ddf3c8ca5834ab015ebafc8b8649ac36ab7?
>>> In terms of when... let's shoot for 2PM PST?
>>> 
>>> -Abe
>>> 
>>> 
>>> On Wed, Jul 23, 2014 at 12:32 PM, Jarek Jarcec Cecho <jarcec@apache.org>
>>> wrote:
>>> 
>>>> I think that both are in. Do let me know when and from which commit you
>>>> want me to branch Abe :-)
>>>> 
>>>> Jarcec
>>>> 
>>>> On Jul 23, 2014, at 10:38 AM, Abraham Elmahrek <abe@cloudera.com> wrote:
>>>> 
>>>>> Hey guys,
>>>>> 
>>>>> As soon as the Ivy and Avro version changes are in, let's start the
>>>>> branching process?
>>>>> 
>>>>> -Abe
>>>>> 
>>>>> 
>>>>> On Wed, Jul 23, 2014 at 1:02 AM, Venkat Ranganathan <
>>>>> vranganathan@hortonworks.com> wrote:
>>>>> 
>>>>>> Generally  the number of tasks is a hint to InputFormat.getSplits()
>>>>>> based on number of splits wanted and it can potentially return
>>>>>> different number of splits.
>>>>>> 
>>>>>> If you see  ExportInputFormat, the split size is determined by
>>>>>> combined file sizes divided by requested num mappers.   This is a
>>>>>> integer division and  there is a potential for getting additional
>>>>>> splits.
>>>>>> 
>>>>>> Thanks
>>>>>> 
>>>>>> Venkat
>>>>>> 
>>>>>> On Tue, Jul 22, 2014 at 11:41 PM, David Robson
>>>>>> <David.Robson@software.dell.com> wrote:
>>>>>>> Yes OraOop is not enabled by default - and also this particular
issue
>>>> is
>>>>>> only when using partitioning - so they would have turned on some
extra
>>>>>> flags. Then they also have to be using 1 more mapper than requested
as
>>>> you
>>>>>> said - so it's a pretty specific combination of events - so I don't
>>>> think
>>>>>> we need to hold up the release for it.
>>>>>>> 
>>>>>>> Do you recall what the behaviour should be in regards to this
- the
>>>>>> documentation leads me to believe if we request 4 mappers we should
>>>> get 4
>>>>>> mappers? If this is only a request and we may get more - perhaps
we
>>>> should
>>>>>> update the documentation about this. Otherwise we could change the
>>>> code to
>>>>>> guarantee no more than 4 mappers?
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Abraham Elmahrek [mailto:abe@cloudera.com]
>>>>>>> Sent: Wednesday, 23 July 2014 3:16 PM
>>>>>>> To: dev@sqoop.apache.org
>>>>>>> Subject: Re: Final Jiras and Release Date
>>>>>>> 
>>>>>>> Hey guys,
>>>>>>> 
>>>>>>> AFAIK we can fall back onto the original connector? With that
being
>>>>>> said, OraOop is an awesome connector... So i'm completely open to
>>>> waiting
>>>>>> for a fix if it isn't too large.
>>>>>>> 
>>>>>>> David, I'm assuming you're referring to the "-m" option? I do
recall
>>>>>> some cases where there may be one more split than desired if splitting
>>>> is
>>>>>> not clean. Hopefully Venkat has more insight!
>>>>>>> 
>>>>>>> -Abe
>>>>>>> 
>>>>>>> 
>>>>>>> On Tue, Jul 22, 2014 at 9:42 PM, David Robson <
>>>>>> David.Robson@software.dell.com> wrote:
>>>>>>> 
>>>>>>>> Hey Venkat,
>>>>>>>> 
>>>>>>>> I had a look into that issue - it seems the problem is even
though
>>>> you
>>>>>>>> request 4 mappers - Sqoop runs 5 mappers. Is the number of
mappers
>>>>>>>> meant to be guaranteed? For example if I say 4 mappers, should
I get
>>>> 4
>>>>>>>> mappers? I guess there could be potential to get less mappers
if the
>>>>>>>> data was for example 1 row - then that would only be processed
by 1
>>>>>>>> mapper. But should it be possible to get more mappers than
requested?
>>>>>>>> 
>>>>>>>> OraOop is assuming there will be no more mappers than requested
- so
>>>>>>>> if this is a valid scenario we would need to modify OraOop.
On the
>>>>>>>> other hand if this is unexpected behaviour then we should
look at why
>>>>>>>> the num mappers is not working?
>>>>>>>> 
>>>>>>>> Either way I don't think we need to fix it for this release
- as it
>>>>>>>> seems customers would be unlikely to hit this issue.
>>>>>>>> 
>>>>>>>> David
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Venkat Ranganathan [mailto:vranganathan@hortonworks.com]
>>>>>>>> Sent: Wednesday, 23 July 2014 1:04 PM
>>>>>>>> To: dev@sqoop.apache.org
>>>>>>>> Subject: Re: Final Jiras and Release Date
>>>>>>>> 
>>>>>>>> Abe
>>>>>>>> 
>>>>>>>> Do we want to target SQOOP-1388 also for 1.4.5 - the one
that Vidya
>>>>>>>> has raised for one of the Oracle connector export failure?
>>>>>>>> 
>>>>>>>> My avro patch is in RB.
>>>>>>>> 
>>>>>>>> Thanks
>>>>>>>> 
>>>>>>>> Venkat
>>>>>>>> 
>>>>>>>> On Tue, Jul 22, 2014 at 6:12 PM, Abraham Elmahrek <abe@cloudera.com>
>>>>>>>> wrote:
>>>>>>>>> Hey guys,
>>>>>>>>> 
>>>>>>>>> It looks like we still have a couple of Jiras still open.
Let's see
>>>>>>>>> how things look tomorrow, but let's have these be the
last two Jiras
>>>>>>>>> slotted for this release.
>>>>>>>>> 
>>>>>>>>> -Abe
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Fri, Jul 18, 2014 at 1:20 PM, Venkat Ranganathan <
>>>>>>>>> vranganathan@hortonworks.com> wrote:
>>>>>>>>> 
>>>>>>>>>> Sounds good.   I have already uploaded the patch
for SQOOP-1358.
>>>> It
>>>>>>>>>> needs to be reviewed by a committer and then committed
if no
>>>>>>>>>> further changes are needed (or more work needed otherwise).
>>>>>>>>>> 
>>>>>>>>>> Thanks for driving the release!
>>>>>>>>>> 
>>>>>>>>>> Venkat
>>>>>>>>>> 
>>>>>>>>>> On Fri, Jul 18, 2014 at 10:58 AM, Jarek Jarcec Cecho
>>>>>>>>>> <jarcec@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>> Seems reasonable to me, thank you for driving
the release Abe!
>>>>>>>>>>> 
>>>>>>>>>>> Jarcec
>>>>>>>>>>> 
>>>>>>>>>>> On Jul 18, 2014, at 10:40 AM, Abraham Elmahrek
<abe@cloudera.com>
>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hey guys,
>>>>>>>>>>>> 
>>>>>>>>>>>> I haven't seen a -1 on this suggestion. So
let's create a
>>>>>>>>>>>> release branch come Wednesday July 23rd.
>>>>>>>>>>>> 
>>>>>>>>>>>> -Abe
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Jul 16, 2014 at 11:38 AM, Abraham
Elmahrek
>>>>>>>>>>>> <abe@cloudera.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Are there any objections to this branch
date?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Abe
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Jul 14, 2014 at 1:22 PM, Abraham
Elmahrek
>>>>>>>>>>>>> <abe@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hey guys,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> It looks like there are a couple
of Jiras that are in progress
>>>>>>>>>>>>>> and slotted for the 1.4.5 release.
If we aim for July 23rd to
>>>>>>>>>>>>>> create a
>>>>>>>>>> release
>>>>>>>>>>>>>> branch, does that give every one
enough time to finish up what
>>>>>>>>>>>>>> they're working on?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> See https://issues.apache.org/jira/browse/SQOOP-1353
for more
>>>>>>>>>>>>>> information.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -Abe
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>>>>> NOTICE: This message is intended for the use of the
individual or
>>>>>>>>>> entity to which it is addressed and may contain information
that is
>>>>>>>>>> confidential, privileged and exempt from disclosure
under
>>>>>>>>>> applicable law. If the reader of this message is
not the intended
>>>>>>>>>> recipient, you are hereby notified that any printing,
copying,
>>>>>>>>>> dissemination, distribution, disclosure or forwarding
of this
>>>>>>>>>> communication is strictly prohibited. If you have
received this
>>>>>>>>>> communication in error, please contact the sender
immediately and
>>>>>>>>>> delete it from your
>>>>>>>> system. Thank You.
>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>>> NOTICE: This message is intended for the use of the individual
or
>>>>>>>> entity to which it is addressed and may contain information
that is
>>>>>>>> confidential, privileged and exempt from disclosure under
applicable
>>>>>>>> law. If the reader of this message is not the intended recipient,
you
>>>>>>>> are hereby notified that any printing, copying, dissemination,
>>>>>>>> distribution, disclosure or forwarding of this communication
is
>>>>>>>> strictly prohibited. If you have received this communication
in
>>>> error,
>>>>>>>> please contact the sender immediately and delete it from
your system.
>>>>>> Thank You.
>>>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>> entity to
>>>>>> which it is addressed and may contain information that is confidential,
>>>>>> privileged and exempt from disclosure under applicable law. If the
>>>> reader
>>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>>> and delete it from your system. Thank You.
>>>>>> 
>>>> 
>>>> 
>>> 
> 


Mime
View raw message