spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charan Adabala <charan.ha...@gmail.com>
Subject Continuously INFO JobScheduler:59 - Added jobs for time *** ms, in my Spark Standalone Cluster.
Date Tue, 29 Mar 2016 10:31:11 GMT
We are working with  Spark Standalone Cluster with 8 Cores and 32GB Ram, with
3 nodes cluster with same configuration.

Some times streaming batch completed in less than 1sec. some times it takes
more than 10 secs at that time below log will appears in console.


2016-03-29 11:35:25,044  INFO TaskSchedulerImpl:59 - Removed TaskSet 18.0,
whose tasks have all completed, from pool 
2016-03-29 11:35:25,044  INFO DAGScheduler:59 - Job 18 finished: foreachRDD
at EventProcessor.java:87, took 1.128755 s
2016-03-29 11:35:31,471  INFO JobScheduler:59 - Added jobs for time
1459231530000 ms
2016-03-29 11:35:35,004  INFO JobScheduler:59 - Added jobs for time
1459231535000 ms
2016-03-29 11:35:40,004  INFO JobScheduler:59 - Added jobs for time
1459231540000 ms
2016-03-29 11:35:45,136  INFO JobScheduler:59 - Added jobs for time
1459231545000 ms
2016-03-29 11:35:50,011  INFO JobScheduler:59 - Added jobs for time
1459231550000 ms
2016-03-29 11:35:55,004  INFO JobScheduler:59 - Added jobs for time
1459231555000 ms
2016-03-29 11:36:00,014  INFO JobScheduler:59 - Added jobs for time
1459231560000 ms
2016-03-29 11:36:05,003  INFO JobScheduler:59 - Added jobs for time
1459231565000 ms
2016-03-29 11:36:10,087  INFO JobScheduler:59 - Added jobs for time
1459231570000 ms
2016-03-29 11:36:15,004  INFO JobScheduler:59 - Added jobs for time
1459231575000 ms
2016-03-29 11:36:20,004  INFO JobScheduler:59 - Added jobs for time
1459231580000 ms
2016-03-29 11:36:25,139  INFO JobScheduler:59 - Added jobs for time
1459231585000 ms


Can you please help, how to solve this problem.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Continuously-INFO-JobScheduler-59-Added-jobs-for-time-ms-in-my-Spark-Standalone-Cluster-tp26621.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message