sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suchit (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-3385) Error while connecting to S3 using Scoop
Date Wed, 19 Sep 2018 19:45:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620949#comment-16620949
] 

Suchit edited comment on SQOOP-3385 at 9/19/18 7:44 PM:
--------------------------------------------------------

Thanks Bogi! I tried above & getting below error. What JAR file I need to for s3a? 

 

Regards

Suchit

18/09/19 15:41:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
18/09/19 15:41:56 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/09/19 15:41:56 INFO manager.OracleManager: Time zone has been set to GMT
Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/AmazonClientException


was (Author: coolsm19):
Thanks Bogi! I tried above & getting below error. What JAR file I need to for s3a? 

 

Regards

Suchit

18/09/19 13:48:20 WARN fs.FileSystem: Cannot load filesystem: java.util.ServiceConfigurationError:
org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.s3a.S3AFileSystem could not
be instantiated
 18/09/19 13:48:20 WARN fs.FileSystem: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
 18/09/19 13:48:20 WARN fs.FileSystem: java.lang.ClassNotFoundException: com.amazonaws.AmazonServiceException
 Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException

18/09/19 13:48:20 WARN fs.FileSystem: Cannot load filesystem: java.util.ServiceConfigurationError:
org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.s3a.S3AFileSystem could not
be instantiated
 18/09/19 13:48:20 WARN fs.FileSystem: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
 18/09/19 13:48:20 WARN fs.FileSystem: java.lang.ClassNotFoundException: com.amazonaws.AmazonServiceException
 Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException

> Error while connecting to S3 using Scoop
> ----------------------------------------
>
>                 Key: SQOOP-3385
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3385
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors
>    Affects Versions: 1.4.7
>            Reporter: Suchit
>            Priority: Minor
>              Labels: S3
>
> I am facing an issue while trying to import file from On Prem DB to S3 using Sqoop.
> Things I an able to do-
> 1- I am connected to S3 , able to run aws s3 ls & other AWS cli commands
> 2- Able to generate a file connecting to DB to local Unix box.
> But when I change the target directory to S3 instead of locat I am getting below error-
>  
>  "ERROR tool.ImportTool: Import failed: AWS Access Key ID and Secret Access Key must
be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId
or fs.s3.awsSecretAccessKey properties (respectively)."
>   
>  Ideally the Sqoop installation should be able to pick up the credentials from credential
file inside .aws directory of the user running the command but  Is there a way I can specify
the credentials?
>   
>  Thanks in advance.
>   



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message