sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: does sqoop support SQL_BUFFER_RESULT to help with long running queries?
Date Fri, 06 Jan 2012 07:20:13 GMT
Hi James,
you can bypass this particular problem by specifying your own boundary query using --boundary-query
command line parameter.

Jarcec

On Thu, Jan 05, 2012 at 06:35:12PM -0800, James Warren wrote:
> Hi Arvind -
> 
> Thanks for the suggestions! ?I tried the first option but it
> fortunately failed. ?From the logs:
> 
> 12/01/05 18:24:56 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN(id), MAX(id) FROM (SELECT SQL_BUFFER_RESULT id ......
> 
> and SQL_BUFFER_RESULT apparently only works for top-level select
> queries. ?Nice idea, though.
> 
> Looks like I'm left with option #2. ?More work, but I like getting my
> hands dirty.
> 
> cheers,
> -James
> 
> On Thu, Jan 5, 2012 at 10:08 AM, Arvind Prabhakar <arvind@apache.org> wrote:
> >
> > Hi James,
> >
> > This has not been tested before - but you could possibly use the
> > --query option to supply your own query that contains the necessary
> > optimization hints. If you find that not working, or limiting in some
> > way - you could also try extending the
> > org.apache.sqoop.manager.MySQLManager implementation to create your
> > own connector that does understand such optimizations.
> >
> > Thanks,
> > Arvind
> >
> > On Wed, Jan 4, 2012 at 4:45 PM, James Warren
> > <james.warren@stanfordalumni.org> wrote:
> > > Greetings -
> > >
> > > Was wondering / hoping if it were possible to configure sqoop to use
> > > SQL_BUFFER_RESULT in the generated select queries. ?From my understanding
> > > sqoop uses READ_COMMITTED as the isolation level, but I'd ideally like to
> > > release the write locks as soon as possible to help alleviate any
> > > replication lag.
> > >
> > > Many thanks,
> > > -jw

Mime
View raw message