sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Han Sen Tey <axemen...@yahoo.com>
Subject SQOOP export from Hadoop Hive to MySQL
Date Tue, 21 May 2013 12:56:16 GMT
Greeting experts,

   I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL. 
Similarly I can find this table in hdfs -- 
/user/hive/warehouse/chrysler_anametrix_aggregated.


   I can successfully execute the following SQOOP command from namenode CLI -

   sqoop export --verbose --connect 
jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table 
CHRYSLER_Anametrix_Aggregated --update-key 
VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession 
--update-mode allowinsert --export-dir 
/user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1 
--input-fields-terminated-by '|' --input-null-string '\\N' 
--input-null-non-string '\\N'

   However, when I want to run the above as part of Oozie workflow action ... 


    <action name="sqoop-node">
        <sqoop xmlns="uri:oozie:sqoop-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
           
 <configuration>
                <property>
                    <name>sqoop.connection.factories</name>
                    <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
                </property>
                <property>
                   
 <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
                <property>
                    <name>mapred.compress.map.output</name>
                    <value>true</value>
                </property>
               
 <property>
                    <name>oozie.service.WorkflowAppService.system.libpath</name>
                    <value>${nameNode}/user/oozie/share/lib/sqoop</value>
                </property>
            </configuration>
           
 <command>export --verbose --connect 
jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table 
CHRYSLER_Anametrix_Aggregated --update-key 
VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession 
--update-mode allowinsert --export-dir
 /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
 --input-fields-terminated-by '|' --input-null-string '\\N' 
--input-null-non-string '\\N'</command>
        </sqoop>
        <ok to="end"/>
        <error to="fail"/>
    </action>

   I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar" shared
library - /user/oozie/share/lib/sqoop.

   Hadoop version - 2.00 - CDH4.2.0
   Sqoop version - 1.4.2. - CDH4.2.0


   Can you experts please guide me  ... a rookie to Hadoop and its projects.

2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394
WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree:
setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task:
 Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea
2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5
2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
 is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter name
instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21
00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the
environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool:
Enabled debug logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions: Character
argument '|' has multiple characters; only the first will be used. 2013-05-21 00:48:40,084
DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories is set; ignoring managers.d
2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
2013-05-21 00:48:40,140
 DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 2013-05-21
00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming
resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated ConnManager
org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool:
Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager:
No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911
DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21
00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.*
FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter:
selected columns: 2013-05-21 00:48:40,931 DEBUG
 org.apache.sqoop.orm.ClassWriter:   VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorMonth
2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorYear 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:   VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   DeviceType 2013-05-21
00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   DeviceBrowser 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:   TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
  SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   BuyID 2013-05-21
00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:   AdvertisementID 2013-05-21 00:48:40,931
DEBUG org.apache.sqoop.orm.ClassWriter:  
 PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   CreativeID
2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   PaidKeyword 2013-05-21 00:48:40,934
DEBUG org.apache.sqoop.orm.ClassWriter:   Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:
  Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   Model300
2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   Model300TownCountry 2013-05-21
00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOLineup 2013-05-21 00:48:40,935 DEBUG
org.apache.sqoop.orm.ClassWriter:   BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:
  BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOOptions 2013-05-21
00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BMOSummary 2013-05-21 00:48:40,935
DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureLineup 2013-05-21
 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureForm 2013-05-21 00:48:40,935
DEBUG org.apache.sqoop.orm.ClassWriter:   BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:
  BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   PreDL
2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   DealerLocate 2013-05-21
00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter:   DLHours 2013-05-21 00:48:40,935 DEBUG
org.apache.sqoop.orm.ClassWriter:   DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   PreSNI 2013-05-21
00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   SearchNewInventory 2013-05-21 00:48:40,936
DEBUG org.apache.sqoop.orm.ClassWriter:   SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  SNIDetails 2013-05-21 00:48:40,936 DEBUG
 org.apache.sqoop.orm.ClassWriter:   SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:
  TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   TestDriveComplete
2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter:   Inventives 2013-05-21 00:48:40,937
DEBUG org.apache.sqoop.orm.ClassWriter:   PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
  PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   PreGAQ
2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   GetAQuote 2013-05-21 00:48:40,937
DEBUG org.apache.sqoop.orm.ClassWriter:   LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
  Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   PhotosGallery 2013-05-21
00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   Interior 2013-05-21 00:48:40,937 DEBUG
org.apache.sqoop.orm.ClassWriter:  
 Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:   Performance 2013-05-21
00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Safety 2013-05-21 00:48:40,938 DEBUG
org.apache.sqoop.orm.ClassWriter:   Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:
  ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Dimensions
2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter:   Warranty 2013-05-21 00:48:40,965
DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21
00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.*
FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter:
Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
2013-05-21 00:48:40,990 DEBUG
 org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:40,990
DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12, VisitorMonth:12,
VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12, TrafficSource:12,
SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12, PaidKeyword:12, Model200:4,
Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4,
BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4, BrochureDownloadLink:4,
PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4, PreSNI:4, SearchNewInventory:4,
SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4,
PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4, PhotosGallery:4, Interior:4,
Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
 Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter:
sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager:
Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992
INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098
DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with
args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   -sourcepath
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/
2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:   -d
 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:   /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/
2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:   -classpath 2013-05-21
00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:  
 /run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
2013-05-21 00:48:48,210 DEBUG
 org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca
2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class
-> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager:
Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar
2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert
functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager:
DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN
org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent
on column 2013-05-21 00:48:48,243 WARN
 org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please
see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation
for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase:
Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase:
Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575
DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
 file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar
2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May
not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient:
Use GenericOptionsParser for parsing the arguments. Applications
 should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat:
Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat:
Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat:
  Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:;
2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job: job_201305131705_0182
 2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient:  map 0% reduce 0% 2013-05-21
00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0,
Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id :
attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient:
Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO
org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015
INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient:
  Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient:     Failed
map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient:     Launched
map tasks=4 2013-05-21 00:49:42,023 INFO
 org.apache.hadoop.mapred.JobClient:     Data-local map tasks=4 2013-05-21 00:49:42,024 INFO
org.apache.hadoop.mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=50187
2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient:     Total time spent by all
reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient:
    Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024
INFO org.apache.hadoop.mapred.JobClient:     Total time spent by all reduces waiting after
reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters
is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033
INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec)
2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated.
 Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase:
Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error
during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task:
Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21
00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0
is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter:
Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output
2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0'
done. 2013-05-21 00:49:43,309 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
 Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1

Regards,
Andy Tey
Student !!
Mime
View raw message