sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Issue in Oozie sqoop action with <command></command> and --query
Date Mon, 22 Oct 2012 15:40:20 GMT
Hi Nitin,
it can actually get quite tricky to correctly split the command line when Oozie would allow
all soft of different escaping inside <command> argument. That's why it fits for simple
use cases and that's why there is tag <arg> for more advance use cases.

I can see why you need to generate your whole command line in your use case. I've actually
done something similar (conditional sqoop command line generation) in the past, but I've used
<arg> tags and just filled argument values (password, username, query). Is something
like that possible in your use case?

Jarcec

On Mon, Oct 22, 2012 at 11:11:21AM -0400, Nitin kak wrote:
> Okay. So the thing is that my pre-compiler could have generated the whole
> command(which requires the --query "select a, b as c from .... " ) as a
> parameter. While what I now need is many more parameter values for
> --connect, --query, --username, --password and all.
> 
> What I feel is that if sqoop command supports "--query" construct, then we
> should be able to use that construct in <command></command> in Oozie.
> 
> Please let me know what you think.
> 
> Regards,
> Nitin
> 
> On Mon, Oct 22, 2012 at 11:01 AM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:
> 
> > Hi Nitin,
> > I personally do not see that as an issue. Tag <command> is meant for
> > simple sqoop command lines and there are tags <arg> for more advance use
> > cases. You happen to be in advance use case and therefore you need <arg>
> > tags. Seems a perfectly valid solution to me.
> >
> > Would you mind describe why do you necessarily need <command> tag and why
> > you can't use <arg> tags?
> >
> > Jarcec
> >
> > On Mon, Oct 22, 2012 at 10:40:18AM -0400, Nitin kak wrote:
> > > Hi Jarcec,
> > >
> > >      Thanks for ur help. Isn't that kind of a issue that we can't use
> > > "--query" in <command><command> in Oozie. If that is, can we raise
this
> > > issue somewhere?
> > >
> > > Regards,
> > > Nitin Kak
> > >
> > > On Fri, Oct 19, 2012 at 5:39 PM, Jarek Jarcec Cecho <jarcec@apache.org
> > >wrote:
> > >
> > > > That is correct - you can't use <command> tag and <arg> tags
together.
> > You
> > > > can use either <command> or <arg>.
> > > >
> > > > Jarcec
> > > >
> > > > On Fri, Oct 19, 2012 at 05:32:12PM -0400, Nitin kak wrote:
> > > > > Ahh...that sucks..I hope they will come up with something in future
> > > > > releases...
> > > > >
> > > > > One follow up question...We can't use <command></command>
in
> > combination
> > > > > with <arg></arg> right?
> > > > >
> > > > > On Fri, Oct 19, 2012 at 5:25 PM, Jarek Jarcec Cecho <
> > jarcec@apache.org
> > > > >wrote:
> > > > >
> > > > > > Hi Nikin,
> > > > > > workaround to use --query parameter inside <command> tag
in Oozie
> > is to
> > > > > > use multiple <arg> tags instead :-)
> > > > > >
> > > > > > Basically Oozie will blindly divide value supplied in <command>
> > tag by
> > > > > > spaces without taking into account any escaping. You are required
> > to
> > > > use
> > > > > > <arg> tags instead of <command> in case that you
need to specify
> > > > arguments
> > > > > > with spaces. For more information please consider Oozie Sqoop
> > action
> > > > > > documentation [1].
> > > > > >
> > > > > > Jarcec
> > > > > >
> > > > > > Links:
> > > > > > 1:
> > > > > >
> > > >
> > http://oozie.apache.org/docs/3.2.0-incubating/DG_SqoopActionExtension.html
> > > > > >
> > > > > > On Fri, Oct 19, 2012 at 05:10:07PM -0400, Nitin kak wrote:
> > > > > > > Hi guys,
> > > > > > >
> > > > > > >       Just wanted to know how we can use "--query" attribute
in
> > > > > > > <command></command> of sqoop action in Oozie.
The issue is that
> > if we
> > > > > > have
> > > > > > > something like
> > > > > > >
> > > > > > > * <command>blah blah ....  --query "select * from
> > table"</command>
> > > > *is
> > > > > > > converted to
> > > > > > >
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>--query</arg>
> > > > > > > <arg>"select</arg>
> > > > > > > <arg>*</arg>
> > > > > > > <arg>from</arg>
> > > > > > > ....so on
> > > > > > > while what we want is
> > > > > > >
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>--query</arg>
> > > > > > > <arg>"select * from table:</arg>
> > > > > > >
> > > > > > >
> > > > > > > Is there a workaround?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Nitin Kak
> > > > > >
> > > >
> >

Mime
View raw message