On Thu, Mar 9, 2017 at 6:27 PM, Burak Ongun <burakongun@gmail.com> wrote:

I encrypt file with openssl then put it on HDFS, I used AES/ECB, 128 bits and salt option, and with some research I find out openssl uses PKCS5 padding as default which are all defaults in CryptoFileLoader class here; 



Here is my encryption process:

    # echo -n "password" > .pw

    # openssl enc -aes-128-ecb -salt -in .pw -out .pw.enc

    # hdfs dfs -put .opw.enc /user/user1/

Sqoop version is 1.4.6


    sqoop import \

    -Dorg.apache.sqoop.credentials.loader.class=org.apache.sqoop.util.password.CryptoFileLoader \

    -Dorg.apache.sqoop.credentials.loader.crypto.passphrase=sqoop \

    --connect jdbc:oracle:thin:@host/database \

    --username user1 \

    --password-file /user/user1/.pw.enc \

    --table db.table1 \

    --hive-import \

    --hive-overwrite \

    --hive-table hivedb.table1 \


which gives: 

    17/03/08 15:10:37 WARN tool.BaseSqoopTool: Failed to load password file

    java.io.IOException: Can't decrypt the password

            at org.apache.sqoop.util.password.CryptoFileLoader.loadPassword(CryptoFileLoader.java:151)

            at org.apache.sqoop.util.CredentialsUtil.fetchPasswordFromLoader(CredentialsUtil.java:81)

            at org.apache.sqoop.util.CredentialsUtil.fetchPassword(CredentialsUtil.java:66)

            at org.apache.sqoop.tool.BaseSqoopTool.applyCredentialsOptions(BaseSqoopTool.java:1042)

            at org.apache.sqoop.tool.BaseSqoopTool.applyCommonOptions(BaseSqoopTool.java:997)

            at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:875)

            at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)

            at org.apache.sqoop.Sqoop.run(Sqoop.java:131)

            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

            at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

            at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

            at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

    Caused by: javax.crypto.BadPaddingException: Given final block not properly padded

            at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:966)

            at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:824)

            at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:436)

            at javax.crypto.Cipher.doFinal(Cipher.java:2165)

            at org.apache.sqoop.util.password.CryptoFileLoader.loadPassword(CryptoFileLoader.java:149)

            ... 12 more

    Error while loading password file: Can't decrypt the password

I tried manually giving the other CryptoFileLoader parameters too and also passing local file to the --password-file .

 I also tried using other algorithms such as CBC but they want parameters (IV, key etc.) and I couldn't pass them because it isn't defined how to in the CryptoFileLoader. 

I can decrypt the file back successfully with openssl. I can't decrypt with Java program(?)

I saw there is an issue with padding but I didn't know what it is and how to encrypt the file with a certain padding method or whatever else to do, I'm not experienced with encryption.

There is also org.apache.sqoop.credentials.loader.crypto.iterations parameter in the class which indicates number of PBKDF2 iterations but I don't know if it changes anything.

Thanks for any help.