sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Syed Akram <akram.ba...@zohocorp.com>
Subject Re: Sqoop import job submit mapreduce stuck at 86%
Date Thu, 05 Mar 2015 17:38:15 GMT
Hi Vineet,

Basically i'm using sqoop2(1.99.3), iam not setting any num of mappers here


Please suggest me the configurations in sqoop2 , to do larger tables import fastly to HDFS.


Setting database server data retrieval timeout : how to set this in sqoop2?
When i'm running small tables, its finishing quickly, But when i'm doing larger tables with
millions of rows, 
reaching mapreduce progress to 86% quickly may b less than a minute. 






Need suggestion on MAPReduce configurations
Since machine configurations are


16GB RAM
500GB Disk


Thanks For Quick Reply

---- On Thu, 05 Mar 2015 22:58:22 +0530 Vineet Mishra&lt;clearmidoubt@gmail.com&gt;
wrote ---- 

Hey syed,
 Can you mention the number of mappers you are running the job with?
 It may be the problem with your database server data retrieval timeout limit, try these things.
. .
 Try to set the timeout limit of your respective database to a higher number
 Or
 Try to run your job with max mapper possible on your cluster using argument -m [number of
mapper]
 Hope that should help you out.
 On Mar 5, 2015 4:24 PM, "Syed Akram" &lt;akram.basha@zohocorp.com&gt; wrote:


Hi,

Iam importing table with more than million rows using sqoop2 1.99.3


After creating job iam submitting it, and then mapreduce job status is RUNNING and less than
1min PROGRESS reaches to 86%
and then stuck for ever, sometimes quit with exception TIMEOUT.


please suggest me, what iam doing wrong, and what are the things i hav to check.


Every large table with is stuck up at 86%.


below is the application status


Thanks
Akram Syed

















All Applications

 


 
Cluster
About
Nodes
ApplicationsNEW
NEW_SAVING
SUBMITTED
ACCEPTED
RUNNING
FINISHED
FAILED
KILLED

Scheduler
Tools

 Cluster Metrics
Apps Submitted Apps Pending Apps Running Apps Completed Containers Running Memory Used Memory
Total Memory Reserved VCores Used VCores Total VCores Reserved Active Nodes Decommissioned
Nodes Lost Nodes Unhealthy Nodes Rebooted Nodes 
1 0 1 0 2 9.84 GB 39.41 GB 0 B 2 40 0 5 0 0 0 0 

Show 20406080100 entries
Search: 


ID
 User
 Name
 Application Type
 Queue
 StartTime
 FinishTime
 State
 FinalStatus
 Progress
 Tracking UI
 
application_1425550787783_0001 sas Sqoop: ImportJob MAPREDUCE default Thu, 05 Mar 2015 10:20:31
GMT N/A RUNNING UNDEFINED 

 ApplicationMaster 
Showing 1 to 1 of 1 entries
FirstPrevious1NextLast


 



 






Mime
View raw message