sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhou, Richard" <richard.z...@intel.com>
Subject RE: Sqoop2 job hung for long time
Date Tue, 02 Jun 2015 07:27:53 GMT
Hey, it seems that resource manager cannot create container.
For the log file in

/var/log/hadoop-yarn
/var/log/sqoop2

You cannot delete the log file manually. CDH will append log into the log file, but if there
is no log file existing in the folder, CDH will not create a new log file.
You will find error message in CM (cluster->yarn->resource manager->log file link
in summary tab).
You need to touch a new log file and change owner to yarn and change mode to 644.

[root@server-654 richard]# ll /var/log/hadoop-yarn/hadoop-cmf-yarn2-RESOURCEMANAGER-server-654.novalocal.log.out
-rw-r--r-- 1 yarn yarn 54243 Jun  2 14:42 /var/log/hadoop-yarn/hadoop-cmf-yarn2-RESOURCEMANAGER-server-654.novalocal.log.out


Regards
Richard


-----Original Message-----
From: SHANKAR REDDY [mailto:sankara.telukutla@gmail.com] 
Sent: Tuesday, June 02, 2015 12:51 PM
To: dev@sqoop.apache.org
Subject: Re: Sqoop2 job hung for long time

Hi Richard,
Please see below.

sqoop:000> set option -name verbose -value true Verbose option was changed to true sqoop:000>
sqoop:000> start job -jid 2 -s Submission details Job ID: 2 Server URL: http://localhost:12000/sqoop/
Created by: ubuntu Creation date: 2015-06-02 04:41:56 UTC Lastly updated by: ubuntu External
ID: job_1433186285881_0007

http://ip-172-31-1-201.us-west-1.compute.internal:8088/proxy/application_1433186285881_0007/
Source Connector schema: Schema{name=clp_sandbox.HADOOP_TEST,columns=[

FixedPoint{name=SERIAL_NO,nullable=true,type=FIXED_POINT,byteSize=4,signed=true},

FixedPoint{name=EMPLOYEE_ID,nullable=true,type=FIXED_POINT,byteSize=4,signed=true},
        Text{name=NAME,nullable=true,type=TEXT,charSize=null}]}
2015-06-02 04:41:56 UTC: BOOTING  - Progress is not available
2015-06-02 04:42:07 UTC: BOOTING  - 0.00 %
2015-06-02 04:42:17 UTC: BOOTING  - 0.00 %
2015-06-02 04:42:27 UTC: BOOTING  - 0.00 %
2015-06-02 04:42:37 UTC: BOOTING  - 0.00 %
2015-06-02 04:42:47 UTC: BOOTING  - 0.00 %
2015-06-02 04:42:57 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:07 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:17 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:27 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:37 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:47 UTC: BOOTING  - 0.00 %
2015-06-02 04:43:57 UTC: BOOTING  - 0.00 %
2015-06-02 04:44:07 UTC: BOOTING  - 0.00 %
2015-06-02 04:44:17 UTC: BOOTING  - 0.00 %
2015-06-02 04:44:27 UTC: BOOTING  - 0.00 %


..
I cleaned up the logs and before doing the above steps and found that no new logs are created
at below location.

/var/log/hadoop-yarn
/var/log/sqoop2


Is there anything I can verify?

Kind Regards,
Sankara Telukutla


On Mon, Jun 1, 2015 at 8:10 PM, Zhou, Richard <richard.zhou@intel.com>
wrote:

> Hey, would you send out the log, including the Sqoop log and Yarn log. 
> It should be in /var/log/ if Cloudera 5.4.1 is installed.
> And run command "set option -name verbose -value true" to set verbose, 
> then re-run the job with "start job -jid 2 -s" to show more information.
>
>
> Regards
> Richard
>
> -----Original Message-----
> From: SHANKAR REDDY [mailto:sankara.telukutla@gmail.com]
> Sent: Tuesday, June 02, 2015 9:16 AM
> To: dev@sqoop.apache.org
> Subject: Sqoop2 job hung for long time
>
> Team,
>
> I have a job which is from MySql to HDFS transformation using sqoop2. 
> The job I have started not responding for long time and seems like 
> there is a problem with YARN which couldn't able to pick up the same. 
> Could you please help me to rectify this problem?
>
> Version details:
> SQOOP2  : 1.99.5
> Cloudera :  5.4.1
>
> application_1433186285881_0004
> <
> http://ec2-52-8-94-128.us-west-1.compute.amazonaws.com:8088/cluster/ap
> p/application_1433186285881_0004
> >
> sqoop2Sqoop: Test Job-copyMAPREDUCEroot.sqoop2Mon Jun 1 17:18:39 -0700 
> 2015
>
> N/AACCEPTEDUNDEFINED
> UNASSIGNED
> <http://ec2-52-8-94-128.us-west-1.compute.amazonaws.com:8088/cluster/#
> >
>
>
> And the job information :
> sqoop:000> show job -jid 2
> 1 job(s) to show:
> Job with id 2 and name Test Job-copy (Enabled: true, Created by null 
> at
> 5/21/15 8:45 AM, Updated by null at 6/2/15 12:18 AM) Using link id 1 
> and Connector id 4
>   From database configuration
>     Schema name: clp_sandbox
>     Table name: HADOOP_TEST
>     Table SQL statement:
>     Table column names:
>     Partition column name: EMPLOYEE_ID
>     Null value allowed for the partition column: true
>     Boundary query:
>   Throttling resources
>     Extractors: 2
>     Loaders: 1
>   ToJob configuration
>     Override null value: false
>     Null value:
>     Output format: SEQUENCE_FILE
>     Compression format: NONE
>     Custom compression format:
>     Output directory: /hadooptest
>
>
>
> Let me know if you need any information I can provide.
>
>
>
> -Shankar
>
Mime
View raw message