sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Senthil Ganesh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SQOOP-3065) Sqoop Import fails when column containing unicode data for split-by
Date Thu, 01 Dec 2016 07:01:58 GMT
Senthil Ganesh created SQOOP-3065:
-------------------------------------

             Summary: Sqoop Import fails when column containing unicode data for split-by
                 Key: SQOOP-3065
                 URL: https://issues.apache.org/jira/browse/SQOOP-3065
             Project: Sqoop
          Issue Type: Bug
          Components: connectors/sqlserver
            Reporter: Senthil Ganesh


sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true --connect jdbc:sqlserver://10.65.141.234;databaseName=sqoop_adapter
--password ****** --map-column-java LANGUAGE=String,CODEPAGE=String,C_NAME_JAP_名前=String,C_ADD_GREEK_ΔΙΕΎΘΥΝΣΗ=String,C_MKTSGMT_CHINESE_市場=String,C_COMMENT_KOREAN_덧글=String
--username sq_admin --as-avrodatafile --split-by C_MKTSGMT_CHINESE_市場 --columns LANGUAGE,CODEPAGE,C_NAME_JAP_名前,C_ADD_GREEK_ΔΙΕΎΘΥΝΣΗ,C_MKTSGMT_CHINESE_市場,C_COMMENT_KOREAN_덧글
--table 日本語_ΕΛΛΗΝΙΚΑ --target-dir hdfs://asd/blaze/workdir/4133141082177241108/W4133141082177241108_Read_日本語_ΕΛΛΗΝΙΚΑ_m_export_日本語_ΕΛΛΗΝΙΚΑ/staged_files




overrides a deprecated API.
2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Note: Recompile
with -Xlint:deprecation for details.
2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Writing jar
file: /tmp/sqoop-adpqa/compile/9013a77a704c396d503bc9c56a95ba1e/日本語_ΕΛΛΗΝΙΚΑ.jar
2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Beginning
import of 日本語_ΕΛΛΗΝΙΚΑ
2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] mapred.jar
is deprecated. Instead, use mapreduce.job.jar
2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Executing
SQL statement: SELECT t.* FROM [日本語_ΕΛΛΗΝΙΚΑ] AS t WHERE 1=0
2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Executing
SQL statement: SELECT t.* FROM [日本語_ΕΛΛΗΝΙΚΑ] AS t WHERE 1=0
2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Writing Avro
schema file: /tmp/sqoop-adpqa/compile/9013a77a704c396d503bc9c56a95ba1e/日本語_ΕΛΛΗΝΙΚΑ.avsc
2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] mapred.map.tasks
is deprecated. Instead, use mapreduce.job.maps
2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] SQOOP_HOME
is unset. May not be able to find all job dependencies.
2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException
as:adpqa (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Exception
encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException
as:adpqa (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error
2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Failing over
to rm71
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Using read
commited transaction isolation
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] BoundingValsQuery:
SELECT MIN([C_MKTSGMT_CHINESE_市場]), MAX([C_MKTSGMT_CHINESE_市場]) FROM [日本語_ΕΛΛΗΝΙΚΑ]
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Generating
splits for a textual index column.
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] If your database
sorts in a case-insensitive order, this may result in a partial import or duplicate records.
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] You are strongly
encouraged to choose an integral split column.
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Cleaning
up the staging area /tmp/hadoop-yarn/staging/adpqa/.staging/job_1480054223215_0084
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException
as:adpqa (auth:PROXY) via adpqa (auth:KERBEROS) cause:java.io.IOException: 0.0015106201171875
is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Encountered
IOException running import job: java.io.IOException: 0.0015106201171875 is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:204)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at java.security.AccessController.doPrivileged(Native
Method)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at javax.security.auth.Subject.doAs(Subject.java:422)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at java.security.AccessController.doPrivileged(Native
Method)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at javax.security.auth.Subject.doAs(Subject.java:422)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:203)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:176)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:273)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:163)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at java.lang.reflect.Method.invoke(Method.java:497)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at com.infa.adapter.sqoop.client.Client.invoke(Client.java:86)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at com.infa.adapter.sqoop.client.Client.main(Client.java:40)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at java.lang.reflect.Method.invoke(Method.java:497)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Caused by:
0.0015106201171875 is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:157)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:116)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:201)
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]      ...
33 more
2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762]



Thanks & Regards,
Rohith



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message