spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Monitoring Spark Jobs
Date Sun, 07 Jun 2015 16:16:21 GMT
It could be a CPU, IO, Network bottleneck, you need to figure out where
exactly its chocking. You can use certain monitoring utilities (like top)
to understand it better.

Thanks
Best Regards

On Sun, Jun 7, 2015 at 4:07 PM, SamyaMaiti <samya.maiti2012@gmail.com>
wrote:

> Hi All,
>
> I have a Spark SQL application to fetch data from Hive, on top I have a
> akka
> layer to run multiple Queries in parallel.
>
> *Please suggest a mechanism, so as to figure out the number of spark jobs
> running in the cluster at a given instance of time. *
>
> I need to do the above as, I see the average response time increasing with
> increase in number of requests, in-spite of increasing the number of cores
> in the cluster. I suspect there is a bottleneck somewhere else.
>
> Regards,
> Sam
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Monitoring-Spark-Jobs-tp23193.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message