sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: does sqoop support SQL_BUFFER_RESULT to help with long running queries?
Date Sun, 08 Jan 2012 09:39:05 GMT
You're welcome sir. I'm glad that sqoop is working for you as you need after all :-)

Jarcec

On Sun, Jan 08, 2012 at 01:20:10AM -0800, James Warren wrote:
> Finally got around to testing this earlier today and it ran smooth as silk.
>  I really should browse the JIRAs more often.  Thanks for the advice (and
> patch), Jarcec!
> 
> cheers,
> -James
> 
> On Thu, Jan 5, 2012 at 11:20 PM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:
> 
> > Hi James,
> > you can bypass this particular problem by specifying your own boundary
> > query using --boundary-query command line parameter.
> >
> > Jarcec
> >
> > On Thu, Jan 05, 2012 at 06:35:12PM -0800, James Warren wrote:
> > > Hi Arvind -
> > >
> > > Thanks for the suggestions! ?I tried the first option but it
> > > fortunately failed. ?From the logs:
> > >
> > > 12/01/05 18:24:56 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> > > SELECT MIN(id), MAX(id) FROM (SELECT SQL_BUFFER_RESULT id ......
> > >
> > > and SQL_BUFFER_RESULT apparently only works for top-level select
> > > queries. ?Nice idea, though.
> > >
> > > Looks like I'm left with option #2. ?More work, but I like getting my
> > > hands dirty.
> > >
> > > cheers,
> > > -James
> > >
> > > On Thu, Jan 5, 2012 at 10:08 AM, Arvind Prabhakar <arvind@apache.org>
> > wrote:
> > > >
> > > > Hi James,
> > > >
> > > > This has not been tested before - but you could possibly use the
> > > > --query option to supply your own query that contains the necessary
> > > > optimization hints. If you find that not working, or limiting in some
> > > > way - you could also try extending the
> > > > org.apache.sqoop.manager.MySQLManager implementation to create your
> > > > own connector that does understand such optimizations.
> > > >
> > > > Thanks,
> > > > Arvind
> > > >
> > > > On Wed, Jan 4, 2012 at 4:45 PM, James Warren
> > > > <james.warren@stanfordalumni.org> wrote:
> > > > > Greetings -
> > > > >
> > > > > Was wondering / hoping if it were possible to configure sqoop to
use
> > > > > SQL_BUFFER_RESULT in the generated select queries. ?From my
> > understanding
> > > > > sqoop uses READ_COMMITTED as the isolation level, but I'd ideally
> > like to
> > > > > release the write locks as soon as possible to help alleviate any
> > > > > replication lag.
> > > > >
> > > > > Many thanks,
> > > > > -jw
> >

Mime
View raw message