sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua Stanfield (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-2282) Add validation check for --hive-import and --append
Date Thu, 23 Jul 2015 23:15:04 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639655#comment-14639655
] 

Joshua Stanfield edited comment on SQOOP-2282 at 7/23/15 11:14 PM:
-------------------------------------------------------------------

Warning message should probably say 
{noformat} 
Please remove the parameter --apend"
{noformat}
 not 
{noformat} "--apend-mode" {noformat}, to reflect the actual parameter name (might be a good
plan to reference static variable APPEND_ARG instead of hard coding the parameter name)

This is also a breaking change for existing jobs that have had both these options set. Probably
worth noting that users using the sqoop metastore may need to recompile their jobs.

Can you explain how these two options are not working together currently? Perhaps it wasn't
working as designed, but silently failed? We have 487 jobs that are running with both of these
options set on cdh 5.1.3 sqoop (1.4.4 + extras) without issues. Testing out a new version
of sqoop (cdh 5.4.4 includes 1.4.5 + this patch) immediately breaks all our jobs. 

Removing the "--append" option does result in a properly running job, and is how I'm planning
on fixing this. And that requires going into the hsql db and manually editing every job (setting
"hdfs.append.dir" to false). But - was just wondering what the original problem was. 


was (Author: p5k6@yahoo.com):
Warning message should probably say "Please remove the parameter `--apend`" not "--apend-mode",
to reflect the actual parameter name (might be a good plan to reference static variable APPEND_ARG
instead of hard coding the parameter name)

This is also a breaking change for existing jobs that have had both these options set. Probably
worth noting that users using the sqoop metastore may need to recompile their jobs.

Can you explain how these two options are not working together currently? Perhaps it wasn't
working as designed, but silently failed? We have 487 jobs that are running with both of these
options set on cdh 5.1.3 sqoop (1.4.4 + extras) without issues. Testing out a new version
of sqoop (cdh 5.4.4 includes 1.4.5 + this patch) immediately breaks all our jobs. 

Removing the "--append" option does result in a properly running job, and is how I'm planning
on fixing this. And that requires going into the hsql db and manually editing every job (setting
"hdfs.append.dir" to false). But - was just wondering what the original problem was. 

> Add validation check for --hive-import and --append
> ---------------------------------------------------
>
>                 Key: SQOOP-2282
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2282
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Abraham Elmahrek
>            Assignee: Abraham Elmahrek
>             Fix For: 1.4.6
>
>         Attachments: SQOOP-2282.patch
>
>
> They don't work together currently. Though, it seems easy to add.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message