sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix GV <felix.gigu...@mate1inc.com>
Subject Re: Sqoop is at stuck
Date Fri, 26 Apr 2013 13:46:40 GMT
Well, that's probably because what you're sqooping is too small.

144 records for a total of 12 KB !

A map reduce job is a pretty huge overhead for such a tiny task hehe...!

--
Félix

On 2013-04-26, at 3:38, hakki buyukcengiz <buyukcengiz.hakki@gmail.com> wrote:

> Hi,
> 
> I have another problem. The sqoop output does not give the map reduce amounts, it did
not print map %0, but it printed only map %100.
> 
> 
> 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on the command-
> 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
> 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set to GMT
> 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM TEST_STG_HAKKI_4
t WHERE 1=0
> 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
> 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar at: /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
> Note: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java
uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
> 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of TEST_STG_HAKKI_4
> 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set to GMT
> 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
> 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker,
sessionId=
> 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
> 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating mysql-connector-java-5.1.18-bin.jar
in /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating paranamer-2.3.jar
in /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-8994443185246229890
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
as /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
as /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating jackson-core-asl-1.8.8.jar
in /tmp/hadoop-root/mapred/local/archive/-1816448567871051996_796880381_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work--5055758146401190448
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
as /tmp/hadoop-root/mapred/local/archive/-1816448567871051996_796880381_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
as /tmp/hadoop-root/mapred/local/archive/-1816448567871051996_796880381_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating commons-io-1.4.jar
in /tmp/hadoop-root/mapred/local/archive/5529489863911010822_-1978668076_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work--6452587124347627821
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/commons-io-1.4.jar
as /tmp/hadoop-root/mapred/local/archive/5529489863911010822_-1978668076_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/commons-io-1.4.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/commons-io-1.4.jar
as /tmp/hadoop-root/mapred/local/archive/5529489863911010822_-1978668076_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/commons-io-1.4.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating avro-mapred-1.7.3-hadoop2.jar
in /tmp/hadoop-root/mapred/local/archive/-5339208977661302622_622410469_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work--8807763574847158712
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-mapred-1.7.3-hadoop2.jar
as /tmp/hadoop-root/mapred/local/archive/-5339208977661302622_622410469_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-mapred-1.7.3-hadoop2.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-mapred-1.7.3-hadoop2.jar
as /tmp/hadoop-root/mapred/local/archive/-5339208977661302622_622410469_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-mapred-1.7.3-hadoop2.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating sqoop-1.4.2-cdh4.2.0.jar
in /tmp/hadoop-root/mapred/local/archive/4074716922816583024_1260311541_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop-work--9130486721005012007
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
as /tmp/hadoop-root/mapred/local/archive/4074716922816583024_1260311541_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
as /tmp/hadoop-root/mapred/local/archive/4074716922816583024_1260311541_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating jackson-mapper-asl-1.8.8.jar
in /tmp/hadoop-root/mapred/local/archive/3464105439164905714_-2087581921_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-5474013804769163238
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
as /tmp/hadoop-root/mapred/local/archive/3464105439164905714_-2087581921_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
as /tmp/hadoop-root/mapred/local/archive/3464105439164905714_-2087581921_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating ant-eclipse-1.0-jvm1.2.jar
in /tmp/hadoop-root/mapred/local/archive/5712808276749236246_-762243847_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-3968897254049561223
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
as /tmp/hadoop-root/mapred/local/archive/5712808276749236246_-762243847_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
as /tmp/hadoop-root/mapred/local/archive/5712808276749236246_-762243847_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating avro-ipc-1.7.3-tests.jar
in /tmp/hadoop-root/mapred/local/archive/5213647986598038750_-2122882234_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-5928105060890500817
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-ipc-1.7.3-tests.jar
as /tmp/hadoop-root/mapred/local/archive/5213647986598038750_-2122882234_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-ipc-1.7.3-tests.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-ipc-1.7.3-tests.jar
as /tmp/hadoop-root/mapred/local/archive/5213647986598038750_-2122882234_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-ipc-1.7.3-tests.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating jetty-util-6.1.26.jar
in /tmp/hadoop-root/mapred/local/archive/-7708254513334657726_1819575672_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-3731213944503314008
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jetty-util-6.1.26.jar
as /tmp/hadoop-root/mapred/local/archive/-7708254513334657726_1819575672_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jetty-util-6.1.26.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jetty-util-6.1.26.jar
as /tmp/hadoop-root/mapred/local/archive/-7708254513334657726_1819575672_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jetty-util-6.1.26.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating ojdbc5.jar
in /tmp/hadoop-root/mapred/local/archive/-5992657128672011180_1807641162_1856884155/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-287403102540626165
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ojdbc5.jar
as /tmp/hadoop-root/mapred/local/archive/-5992657128672011180_1807641162_1856884155/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ojdbc5.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ojdbc5.jar
as /tmp/hadoop-root/mapred/local/archive/-5992657128672011180_1807641162_1856884155/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ojdbc5.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating snappy-java-1.0.4.1.jar
in /tmp/hadoop-root/mapred/local/archive/-5814335339036157531_-2124538834_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-3525552501228133577
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/snappy-java-1.0.4.1.jar
as /tmp/hadoop-root/mapred/local/archive/-5814335339036157531_-2124538834_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/snappy-java-1.0.4.1.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/snappy-java-1.0.4.1.jar
as /tmp/hadoop-root/mapred/local/archive/-5814335339036157531_-2124538834_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/snappy-java-1.0.4.1.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating netty-3.4.0.Final.jar
in /tmp/hadoop-root/mapred/local/archive/-8676376897584946457_-1136278949_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-7567273204327099009
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/netty-3.4.0.Final.jar
as /tmp/hadoop-root/mapred/local/archive/-8676376897584946457_-1136278949_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/netty-3.4.0.Final.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/netty-3.4.0.Final.jar
as /tmp/hadoop-root/mapred/local/archive/-8676376897584946457_-1136278949_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/netty-3.4.0.Final.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating ant-contrib-1.0b3.jar
in /tmp/hadoop-root/mapred/local/archive/2129323065309420499_676320461_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work--2255937424995958309
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-contrib-1.0b3.jar
as /tmp/hadoop-root/mapred/local/archive/2129323065309420499_676320461_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-contrib-1.0b3.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-contrib-1.0b3.jar
as /tmp/hadoop-root/mapred/local/archive/2129323065309420499_676320461_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-contrib-1.0b3.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating servlet-api-2.5-20081211.jar
in /tmp/hadoop-root/mapred/local/archive/-4519017300615876531_-1709987147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work--9042507534758247833
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/servlet-api-2.5-20081211.jar
as /tmp/hadoop-root/mapred/local/archive/-4519017300615876531_-1709987147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/servlet-api-2.5-20081211.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/servlet-api-2.5-20081211.jar
as /tmp/hadoop-root/mapred/local/archive/-4519017300615876531_-1709987147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/servlet-api-2.5-20081211.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating hsqldb-1.8.0.10.jar
in /tmp/hadoop-root/mapred/local/archive/-8062229134739405628_885269916_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-2144687959555476626
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/hsqldb-1.8.0.10.jar
as /tmp/hadoop-root/mapred/local/archive/-8062229134739405628_885269916_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/hsqldb-1.8.0.10.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/hsqldb-1.8.0.10.jar
as /tmp/hadoop-root/mapred/local/archive/-8062229134739405628_885269916_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/hsqldb-1.8.0.10.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating avro-1.7.3.jar
in /tmp/hadoop-root/mapred/local/archive/-4530885538574710561_879602147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-7630716223949344480
with rwxr-xr-x
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-1.7.3.jar
as /tmp/hadoop-root/mapred/local/archive/-4530885538574710561_879602147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-1.7.3.jar
> 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-1.7.3.jar
as /tmp/hadoop-root/mapred/local/archive/-4530885538574710561_879602147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-1.7.3.jar
> 13/04/26 10:10:28 INFO mapred.JobClient: Running job: job_local1889750887_0001
> 13/04/26 10:10:28 INFO mapred.LocalJobRunner: OutputCommitter set in config null
> 13/04/26 10:10:28 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter
> 13/04/26 10:10:28 INFO mapred.LocalJobRunner: Waiting for map tasks
> 13/04/26 10:10:28 INFO mapred.LocalJobRunner: Starting task: attempt_local1889750887_0001_m_000000_0
> 13/04/26 10:10:28 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
> 13/04/26 10:10:28 INFO util.ProcessTree: setsid exited with exit code 0
> 13/04/26 10:10:28 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1c3508c0
> 13/04/26 10:10:28 INFO mapred.MapTask: Processing split: Paths:/user/export_import/SQOOP/part-m-00000:0+431
> 13/04/26 10:10:29 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished.
keepGoing=false
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task:attempt_local1889750887_0001_m_000000_0 is done.
And is in the process of commiting
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task 'attempt_local1889750887_0001_m_000000_0' done.
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Finishing task: attempt_local1889750887_0001_m_000000_0
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Starting task: attempt_local1889750887_0001_m_000001_0
> 13/04/26 10:10:29 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
> 13/04/26 10:10:29 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4ec1429c
> 13/04/26 10:10:29 INFO mapred.MapTask: Processing split: Paths:/user/export_import/SQOOP/part-m-00000:431+431
> 13/04/26 10:10:29 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished.
keepGoing=false
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task:attempt_local1889750887_0001_m_000001_0 is done.
And is in the process of commiting
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task 'attempt_local1889750887_0001_m_000001_0' done.
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Finishing task: attempt_local1889750887_0001_m_000001_0
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Starting task: attempt_local1889750887_0001_m_000002_0
> 13/04/26 10:10:29 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
> 13/04/26 10:10:29 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@40944dbd
> 13/04/26 10:10:29 INFO mapred.MapTask: Processing split: Paths:/user/export_import/SQOOP/part-m-00000:862+431
> 13/04/26 10:10:29 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished.
keepGoing=false
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task:attempt_local1889750887_0001_m_000002_0 is done.
And is in the process of commiting
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task 'attempt_local1889750887_0001_m_000002_0' done.
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Finishing task: attempt_local1889750887_0001_m_000002_0
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Starting task: attempt_local1889750887_0001_m_000003_0
> 13/04/26 10:10:29 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
> 13/04/26 10:10:29 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@78216b59
> 13/04/26 10:10:29 INFO mapred.MapTask: Processing split: Paths:/user/export_import/SQOOP/part-m-00000:1293+431
> 13/04/26 10:10:29 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished.
keepGoing=false
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task:attempt_local1889750887_0001_m_000003_0 is done.
And is in the process of commiting
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner:
> 13/04/26 10:10:29 INFO mapred.Task: Task 'attempt_local1889750887_0001_m_000003_0' done.
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Finishing task: attempt_local1889750887_0001_m_000003_0
> 13/04/26 10:10:29 INFO mapred.LocalJobRunner: Map task executor complete.
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/2129323065309420499_676320461_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-contrib-1.0b3.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/5712808276749236246_-762243847_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-4530885538574710561_879602147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-1.7.3.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/5213647986598038750_-2122882234_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-ipc-1.7.3-tests.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-5339208977661302622_622410469_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/avro-mapred-1.7.3-hadoop2.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/5529489863911010822_-1978668076_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/commons-io-1.4.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-8062229134739405628_885269916_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/hsqldb-1.8.0.10.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-1816448567871051996_796880381_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/3464105439164905714_-2087581921_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-7708254513334657726_1819575672_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/jetty-util-6.1.26.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-8676376897584946457_-1136278949_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/netty-3.4.0.Final.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-5992657128672011180_1807641162_1856884155/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/ojdbc5.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-4519017300615876531_-1709987147_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/servlet-api-2.5-20081211.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/-5814335339036157531_-2124538834_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/snappy-java-1.0.4.1.jar
> 13/04/26 10:10:29 INFO filecache.TrackerDistributedCacheManager: Deleted path /tmp/hadoop-root/mapred/local/archive/4074716922816583024_1260311541_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar
> 13/04/26 10:10:29 INFO mapred.JobClient:  map 100% reduce 0%
> 13/04/26 10:10:29 INFO mapred.JobClient: Job complete: job_local1889750887_0001
> 13/04/26 10:10:29 INFO mapred.JobClient: Counters: 18
> 13/04/26 10:10:29 INFO mapred.JobClient:   File System Counters
> 13/04/26 10:10:29 INFO mapred.JobClient:     FILE: Number of bytes read=34008758
> 13/04/26 10:10:29 INFO mapred.JobClient:     FILE: Number of bytes written=34693692
> 13/04/26 10:10:29 INFO mapred.JobClient:     FILE: Number of read operations=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     FILE: Number of large read operations=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     FILE: Number of write operations=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     HDFS: Number of bytes read=12978
> 13/04/26 10:10:29 INFO mapred.JobClient:     HDFS: Number of bytes written=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     HDFS: Number of read operations=66
> 13/04/26 10:10:29 INFO mapred.JobClient:     HDFS: Number of large read operations=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     HDFS: Number of write operations=0
> 13/04/26 10:10:29 INFO mapred.JobClient:   Map-Reduce Framework
> 13/04/26 10:10:29 INFO mapred.JobClient:     Map input records=144
> 13/04/26 10:10:29 INFO mapred.JobClient:     Map output records=144
> 13/04/26 10:10:29 INFO mapred.JobClient:     Input split bytes=476
> 13/04/26 10:10:29 INFO mapred.JobClient:     Spilled Records=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     CPU time spent (ms)=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     Physical memory (bytes) snapshot=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=0
> 13/04/26 10:10:29 INFO mapred.JobClient:     Total committed heap usage (bytes)=573046784
> 13/04/26 10:10:29 INFO mapreduce.ExportJobBase: Transferred 12.6738 KB in 1.9417 seconds
(6.5273 KB/sec)
> 13/04/26 10:10:29 INFO mapreduce.ExportJobBase: Exported 144 records.
> 
> 
> 
> 2013/4/24 Venkat Ranganathan <vranganathan@hortonworks.com>
>> The information referenced by Jarcec is something that will be valuable.   But one
thing to check is whether you have issues with random generation  that is used internally
by the JDBC driver.   You want to  modify mapred.child.java.opts in mapred-site.xml and add
-Djava.security.egd=file:/dev/./urandom.   That is one common issue with the use SecureRandom
>> 
>> 
>> Thanks
>> 
>> Venkat
>> 
>> 
>> On Wed, Apr 24, 2013 at 8:47 AM, Jarek Jarcec Cecho <jarcec@apache.org> wrote:
>>> Hi sir,
>>> jstack is a standard java tool for exploring running thread of any Java application.
Please take a look on the official documentation for more information [1].
>>> 
>>> In a nutshell:
>>> 1) Reproduce your issue
>>> 2) Get Sqoop PID (process id)
>>> 3) jstack $PID > sqoop.jstack
>>> 4) Send the sqoop.jstack to the mailing list
>>> 
>>> Jarcec
>>> 
>>> Links:
>>> 1: http://docs.oracle.com/javase/6/docs/technotes/tools/share/jstack.html
>>> 
>>> On Wed, Apr 24, 2013 at 06:14:42PM +0300, hakki buyukcengiz wrote:
>>> > Hi Jarek,
>>> >
>>> > What do you mean with jstack? What i had sent was the terminal output of
>>> > sqoop.
>>> >
>>> > Sent from Android
>>> > 24 Nis 2013 17:12 tarihinde "Jarek Jarcec Cecho" <jarcec@apache.org>
yazdı:
>>> >
>>> > > Hi Hakki,
>>> > > would you mind sharing jstack of the stuck process?
>>> > >
>>> > > Jarcec
>>> > >
>>> > > On Wed, Apr 24, 2013 at 09:58:56AM +0300, hakki buyukcengiz wrote:
>>> > > > Hi,
>>> > > >
>>> > > > I had running sqoop export commands like:
>>> > > >
>>> > > > sqoop export --connect 'jdbc:oracle:thin:@<ip>:<port>:<service>'
>>> > > --username
>>> > > > DUMMY --password 'DUMMY' --table TEST_STG --export-dir
>>> > > > /user/export_import/SQOOP --fields-terminated-by '\t'
>>> > > --lines-terminated-by
>>> > > > '\n'
>>> > > >
>>> > > > This command had run successfully but suddenly it become falling
into
>>> > > > stuck. When I start, it prints the normal message like this:
>>> > > >
>>> > > > [root@devnode1 ~]# sqoop export --connect
>>> > > > 'jdbc:oracle:thin:@<ip>:<port>:<service>'
>>> > > > --username DUMMY --password 'DUMMY' --table TEST_STG --export-dir
>>> > > > /user/export_import/SQOOP --fields-terminated-by '\t'
>>> > > --lines-terminated-by
>>> > > > '\n'
>>> > > > 13/04/24 09:46:19 WARN tool.BaseSqoopTool: Setting your password
on the
>>> > > > command-line is insecure. Consider using -P instead.
>>> > > > 13/04/24 09:46:19 INFO manager.SqlManager: Using default fetchSize
of
>>> > > 1000
>>> > > > 13/04/24 09:46:19 INFO tool.CodeGenTool: Beginning code generation
>>> > > > 13/04/24 09:46:31 INFO manager.OracleManager: Time zone has been
set to
>>> > > GMT
>>> > > > 13/04/24 09:46:31 INFO manager.SqlManager: Executing SQL statement:
>>> > > SELECT
>>> > > > t.* FROM TEST_STG t WHERE 1=0
>>> > > > 13/04/24 09:46:31 INFO orm.CompilationManager: HADOOP_MAPRED_HOME
is
>>> > > >
>>> > > /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
>>> > > > 13/04/24 09:46:31 INFO orm.CompilationManager: Found hadoop core
jar at:
>>> > > >
>>> > > /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
>>> > > > Note:
>>> > > > /tmp/sqoop-root/compile/13ac98badc5d4c43ab02815245c500e8/TEST_STG.java
>>> > > uses
>>> > > > or overrides a deprecated API.
>>> > > > Note: Recompile with -Xlint:deprecation for details.
>>> > > > 13/04/24 09:46:33 INFO orm.CompilationManager: Writing jar file:
>>> > > > /tmp/sqoop-root/compile/13ac98badc5d4c43ab02815245c500e8/TEST_STG.jar
>>> > > > 13/04/24 09:46:33 INFO mapreduce.ExportJobBase: Beginning export
of
>>> > > > TEST_STG_HAKKI_4
>>> > > > 13/04/24 09:46:34 INFO manager.OracleManager: Time zone has been
set to
>>> > > GMT
>>> > > >
>>> > > >
>>> > > > And after this point sqoop gives no further message. It inserts
no record
>>> > > > to the db too.
>>> > > > What can be the reason for this?
>>> > > >
>>> > > > Thanks,
>>> > > >
>>> > > > --
>>> > > > Hakki Buyukcengiz
>>> > > > TR
>>> > >
> 
> 
> 
> -- 
> Hakki Buyukcengiz
> TR

Mime
View raw message