sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kaya Kupferschmidt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2194) sqoop start script looks for jars in wrong order
Date Mon, 07 Sep 2015 05:51:45 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733265#comment-14733265
] 

Kaya Kupferschmidt commented on SQOOP-2194:
-------------------------------------------

Thanks for your investigation. Actually the problem happened with Cloudera CDH 5.3.x, but
I cannot reproduce it any more with Sqoop 1.4.5 in Cloudera CDH 5.4.x. I did a lttiel investigations,
and it seems that the mechanism for adding user jars has been changed. Now there is an explicit
SQOOP_USER_CLASSPATH variable, which is evaluated in the script "configure-sqoop". From my
side, you can consider this issue as "fixed".

> sqoop start script looks for jars in wrong order
> ------------------------------------------------
>
>                 Key: SQOOP-2194
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2194
>             Project: Sqoop
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.4.5
>         Environment: Linux
>            Reporter: Kaya Kupferschmidt
>
> The sqoop start script allows the user to include custom jar's by placing them into the
folder "/var/lib/sqoop". In addition to that location, the script also looks into the folder
"/usr/share/java". Technically this is performed by the line
> SQOOP_JARS=`ls /var/lib/sqoop/.jar /usr/share/java/.jar 2>/dev/null`
> in the ssqoop start script. This seems to allow to store sqoop-specific versions of jar
files in "/var/lib/sqoop" which would override versions of similar jars in "/usr/share/java".
For example (this was our use case), this allows to store a more up-to-date version of the
java-mysql-connector in /var/lib/sqoop while the system provided version still remains in
/usr/share/java.
> Unfortunetaly this doesn't work as expected, because the "ls" command will sort its result
by name. So all files in "/usr/share/java" are sorted in front of the files in "/var/lib/sqoop".
An easy fix for the situation is to turn off the sorting by adding the option "-U" to ls:
> SQOOP_JARS=`ls -U /var/lib/sqoop/.jar /usr/share/java/.jar 2>/dev/null`
> I guess the behaviour with this modification reflects the original idea behind that line.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message