sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Issue in Oozie sqoop action with <command></command> and --query
Date Mon, 22 Oct 2012 17:27:22 GMT
Unfortunately escaping with double quotes in <command> tag is not supported. You really
need to use <arg> tag where the escaping do not make sense as each argument is in it's
own tag. For example:

...
  <arg>--query</arg>
  <arg>select * from my_cool_table where $CONDITIONS</arg>
...

Jarcec

On Mon, Oct 22, 2012 at 01:03:25PM -0400, Nitin kak wrote:
> Yup. Its possible.It might sound naive but can we use double quotes(") e.g
> to specify that the string that follows "--query" and is in double quotes
> is just one parameter?
> 
> On Mon, Oct 22, 2012 at 11:40 AM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:
> 
> > Hi Nitin,
> > it can actually get quite tricky to correctly split the command line when
> > Oozie would allow all soft of different escaping inside <command> argument.
> > That's why it fits for simple use cases and that's why there is tag <arg>
> > for more advance use cases.
> >
> > I can see why you need to generate your whole command line in your use
> > case. I've actually done something similar (conditional sqoop command line
> > generation) in the past, but I've used <arg> tags and just filled argument
> > values (password, username, query). Is something like that possible in your
> > use case?
> >
> > Jarcec
> >
> > On Mon, Oct 22, 2012 at 11:11:21AM -0400, Nitin kak wrote:
> > > Okay. So the thing is that my pre-compiler could have generated the whole
> > > command(which requires the --query "select a, b as c from .... " ) as a
> > > parameter. While what I now need is many more parameter values for
> > > --connect, --query, --username, --password and all.
> > >
> > > What I feel is that if sqoop command supports "--query" construct, then
> > we
> > > should be able to use that construct in <command></command> in
Oozie.
> > >
> > > Please let me know what you think.
> > >
> > > Regards,
> > > Nitin
> > >
> > > On Mon, Oct 22, 2012 at 11:01 AM, Jarek Jarcec Cecho <jarcec@apache.org
> > >wrote:
> > >
> > > > Hi Nitin,
> > > > I personally do not see that as an issue. Tag <command> is meant
for
> > > > simple sqoop command lines and there are tags <arg> for more advance
> > use
> > > > cases. You happen to be in advance use case and therefore you need
> > <arg>
> > > > tags. Seems a perfectly valid solution to me.
> > > >
> > > > Would you mind describe why do you necessarily need <command> tag
and
> > why
> > > > you can't use <arg> tags?
> > > >
> > > > Jarcec
> > > >
> > > > On Mon, Oct 22, 2012 at 10:40:18AM -0400, Nitin kak wrote:
> > > > > Hi Jarcec,
> > > > >
> > > > >      Thanks for ur help. Isn't that kind of a issue that we can't
use
> > > > > "--query" in <command><command> in Oozie. If that is,
can we raise
> > this
> > > > > issue somewhere?
> > > > >
> > > > > Regards,
> > > > > Nitin Kak
> > > > >
> > > > > On Fri, Oct 19, 2012 at 5:39 PM, Jarek Jarcec Cecho <
> > jarcec@apache.org
> > > > >wrote:
> > > > >
> > > > > > That is correct - you can't use <command> tag and <arg>
tags
> > together.
> > > > You
> > > > > > can use either <command> or <arg>.
> > > > > >
> > > > > > Jarcec
> > > > > >
> > > > > > On Fri, Oct 19, 2012 at 05:32:12PM -0400, Nitin kak wrote:
> > > > > > > Ahh...that sucks..I hope they will come up with something
in
> > future
> > > > > > > releases...
> > > > > > >
> > > > > > > One follow up question...We can't use <command></command>
in
> > > > combination
> > > > > > > with <arg></arg> right?
> > > > > > >
> > > > > > > On Fri, Oct 19, 2012 at 5:25 PM, Jarek Jarcec Cecho <
> > > > jarcec@apache.org
> > > > > > >wrote:
> > > > > > >
> > > > > > > > Hi Nikin,
> > > > > > > > workaround to use --query parameter inside <command>
tag in
> > Oozie
> > > > is to
> > > > > > > > use multiple <arg> tags instead :-)
> > > > > > > >
> > > > > > > > Basically Oozie will blindly divide value supplied
in <command>
> > > > tag by
> > > > > > > > spaces without taking into account any escaping. You
are
> > required
> > > > to
> > > > > > use
> > > > > > > > <arg> tags instead of <command> in case
that you need to
> > specify
> > > > > > arguments
> > > > > > > > with spaces. For more information please consider
Oozie Sqoop
> > > > action
> > > > > > > > documentation [1].
> > > > > > > >
> > > > > > > > Jarcec
> > > > > > > >
> > > > > > > > Links:
> > > > > > > > 1:
> > > > > > > >
> > > > > >
> > > >
> > http://oozie.apache.org/docs/3.2.0-incubating/DG_SqoopActionExtension.html
> > > > > > > >
> > > > > > > > On Fri, Oct 19, 2012 at 05:10:07PM -0400, Nitin kak
wrote:
> > > > > > > > > Hi guys,
> > > > > > > > >
> > > > > > > > >       Just wanted to know how we can use "--query"
attribute
> > in
> > > > > > > > > <command></command> of sqoop action
in Oozie. The issue is
> > that
> > > > if we
> > > > > > > > have
> > > > > > > > > something like
> > > > > > > > >
> > > > > > > > > * <command>blah blah ....  --query "select
* from
> > > > table"</command>
> > > > > > *is
> > > > > > > > > converted to
> > > > > > > > >
> > > > > > > > > <arg>blah</arg>
> > > > > > > > > <arg>blah</arg>
> > > > > > > > > <arg>--query</arg>
> > > > > > > > > <arg>"select</arg>
> > > > > > > > > <arg>*</arg>
> > > > > > > > > <arg>from</arg>
> > > > > > > > > ....so on
> > > > > > > > > while what we want is
> > > > > > > > >
> > > > > > > > > <arg>blah</arg>
> > > > > > > > > <arg>blah</arg>
> > > > > > > > > <arg>--query</arg>
> > > > > > > > > <arg>"select * from table:</arg>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Is there a workaround?
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Nitin Kak
> > > > > > > >
> > > > > >
> > > >
> >

Mime
View raw message