sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthic Hariharan <khariha...@rmn.com>
Subject Way to pass -D parameters to sqoop merge
Date Wed, 06 May 2015 06:29:44 GMT
I am using sqoop 1.4.5. Since sqoop jobs are essentially jobs built on top
of hadoop I was wondering if there is a way to pass
'-Dproperty.name=property.value' arguments to a sqoop merge command. I have
sqoop running on an Amazon EMR Cluster. My sqoop job gets the incremental
data out and then performs a merge with S3 as the target-dir. But when the
files are created on S3 as a result of the merge command, there are no
access control list permissions set on those files which render them
unusable by anyone with access to that bucket. One way I have noticed that
can be fixed is to set the fs.s3.canned.acl value to FullBucketOwnerControl
when running the job. I can do this trivially when running sqoop import
command. How do I do it for sqoop merge command?


This e-mail, including attachments, contains confidential and/or 
proprietary information, and may be used only by the person or entity to 
which it is addressed. The reader is hereby notified that any 
dissemination, distribution or copying of this e-mail is prohibited. If you 
have received this e-mail in error, please notify the sender by replying to 
this message and delete this e-mail immediately.

View raw message