sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: SQOOP export from Hadoop Hive to MySQL
Date Thu, 23 May 2013 08:45:55 GMT
Hi Han,
would you mind sharing entire Sqoop log generated with parameter --verbose as an attachment?
It might be also beneficial to share the map task log.

Oozie is not using bash nor any other shell to execute Sqoop, so you should remove any escaping
introduced especially for shell. Looking at your example, I think that it should be something
like:

  ... --innput-fields-terminated-by | --input-null-string \\N ...

Jarcec

On Tue, May 21, 2013 at 05:56:16AM -0700, Han Sen Tey wrote:
> Greeting experts,
> 
>    I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL. 
> Similarly I can find this table in hdfs -- 
> /user/hive/warehouse/chrysler_anametrix_aggregated.
> 
> 
>    I can successfully execute the following SQOOP command from namenode CLI -
> 
>    sqoop export --verbose --connect 
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table 
> CHRYSLER_Anametrix_Aggregated --update-key 
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession 
> --update-mode allowinsert --export-dir 
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1 
> --input-fields-terminated-by '|' --input-null-string '\\N' 
> --input-null-non-string '\\N'
> 
>    However, when I want to run the above as part of Oozie workflow action ... 
> 
> 
>     <action name="sqoop-node">
>         <sqoop xmlns="uri:oozie:sqoop-action:0.2">
>             <job-tracker>${jobTracker}</job-tracker>
>             <name-node>${nameNode}</name-node>
>            
>  <configuration>
>                 <property>
>                     <name>sqoop.connection.factories</name>
>                     <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
>                 </property>
>                 <property>
>                    
>  <name>mapred.job.queue.name</name>
>                     <value>${queueName}</value>
>                 </property>
>                 <property>
>                     <name>mapred.compress.map.output</name>
>                     <value>true</value>
>                 </property>
>                
>  <property>
>                     <name>oozie.service.WorkflowAppService.system.libpath</name>
>                     <value>${nameNode}/user/oozie/share/lib/sqoop</value>
>                 </property>
>             </configuration>
>            
>  <command>export --verbose --connect 
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table 
> CHRYSLER_Anametrix_Aggregated --update-key 
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession 
> --update-mode allowinsert --export-dir
>  /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
>  --input-fields-terminated-by '|' --input-null-string '\\N' 
> --input-null-non-string '\\N'</command>
>         </sqoop>
>         <ok to="end"/>
>         <error to="fail"/>
>     </action>
> 
>    I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar"
shared library - /user/oozie/share/lib/sqoop.
> 
>    Hadoop version - 2.00 - CDH4.2.0
>    Sqoop version - 1.4.2. - CDH4.2.0
> 
> 
>    Can you experts please guide me  ... a rookie to Hadoop and its projects.
> 
> 2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394
WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree:
setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task:
 Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea
2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5
2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
>  is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter
name instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks:
0 2013-05-21 00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been
set in the environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032
DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug logging. 2013-05-21 00:48:40,036
WARN org.apache.sqoop.SqoopOptions: Character argument '|' has multiple characters; only the
first will be used. 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories
is set; ignoring managers.d 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded
manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139
DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
2013-05-21 00:48:40,140
>  DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql:
2013-05-21 00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL
streaming resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated
ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool:
Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager:
No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911
DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21
00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.*
FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter:
selected columns: 2013-05-21 00:48:40,931 DEBUG
>  org.apache.sqoop.orm.ClassWriter:   VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorMonth
2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorYear 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   DeviceType 2013-05-21
00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   DeviceBrowser 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:   TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   BuyID 2013-05-21
00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   AdvertisementID 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:  
>  PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   CreativeID
2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   PaidKeyword 2013-05-21 00:48:40,934
DEBUG org.apache.sqoop.orm.ClassWriter:   Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
  Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   Model300
2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   Model300TownCountry 2013-05-21
00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOLineup 2013-05-21 00:48:40,935 DEBUG
org.apache.sqoop.orm.ClassWriter:   BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:
  BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOOptions 2013-05-21
00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOSummary 2013-05-21 00:48:40,935
DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureLineup 2013-05-21
>  00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureForm 2013-05-21 00:48:40,935
DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:
  BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   PreDL
2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   DealerLocate 2013-05-21
00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   DLHours 2013-05-21 00:48:40,935 DEBUG
org.apache.sqoop.orm.ClassWriter:   DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   PreSNI 2013-05-21
00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   SearchNewInventory 2013-05-21 00:48:40,936
DEBUG org.apache.sqoop.orm.ClassWriter:   SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  SNIDetails 2013-05-21 00:48:40,936 DEBUG
>  org.apache.sqoop.orm.ClassWriter:   SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   TestDriveComplete
2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   Inventives 2013-05-21 00:48:40,937
DEBUG org.apache.sqoop.orm.ClassWriter:   PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
  PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   PreGAQ
2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   GetAQuote 2013-05-21 00:48:40,937
DEBUG org.apache.sqoop.orm.ClassWriter:   LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
  Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   PhotosGallery 2013-05-21
00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   Interior 2013-05-21 00:48:40,937 DEBUG
org.apache.sqoop.orm.ClassWriter:  
>  Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   Performance
2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Safety 2013-05-21 00:48:40,938
DEBUG org.apache.sqoop.orm.ClassWriter:   Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:
  ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Dimensions
2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Warranty 2013-05-21 00:48:40,965
DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21
00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.*
FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter:
Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
2013-05-21 00:48:40,990 DEBUG
>  org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21
00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12,
VisitorMonth:12, VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12,
TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12,
PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4,
BMOModels:4, BMOColors:4, BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4,
BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4,
PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4,
Inventives:4, PreCal:4, PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4,
PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
>  Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter:
sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager:
Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992
INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098
DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with
args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   -sourcepath
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   -d
>  2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:   /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/
2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:   -classpath 2013-05-21
00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:  
>  /run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
> 0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
> asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
> reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
> /usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
> usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
> onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
> op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
> p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
> user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
> er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
> l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
2013-05-21 00:48:48,210 DEBUG
>  org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca
2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class
-> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager:
Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert
functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager:
DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN
org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent
on column 2013-05-21 00:48:48,243 WARN
>  org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please
see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation
for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase:
Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase:
Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575
DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
>  file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar
2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May
not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient:
Use GenericOptionsParser for parsing the arguments. Applications
>  should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat:
Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat:
Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
  Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:;
2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job: job_201305131705_0182
>  2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient:  map 0% reduce 0% 2013-05-21
00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0,
Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id :
attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient:
Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO
org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015
INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient:
  Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient:     Failed
map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient:     Launched
map tasks=4 2013-05-21 00:49:42,023 INFO
>  org.apache.hadoop.mapred.JobClient:     Data-local map tasks=4 2013-05-21 00:49:42,024
INFO org.apache.hadoop.mapred.JobClient:     Total time spent by all maps in occupied slots
(ms)=50187 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient:     Total time
spent by all reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient:
    Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024
INFO org.apache.hadoop.mapred.JobClient:     Total time spent by all reduces waiting after
reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters
is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033
INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec)
2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated.
>  Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase:
Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error
during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task:
Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21
00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0
is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter:
Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output
2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0'
done. 2013-05-21 00:49:43,309 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
>  Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 
> Regards,
> Andy Tey
> Student !!

Mime
View raw message