flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andra Lungu <lungu.an...@gmail.com>
Subject Re: extract workers' resources stats
Date Fri, 07 Aug 2015 15:43:50 GMT

You can use something like iostat to extract the CPU usage.
For instance, I call this script on the JM node:
#!/usr/bin/env bash

lines=`cat /home/andra.lungu/hostnames.txt | paste -d, -s`
echo "${lines}"
IFS="," read -ra hostnames <<<"${lines}"
for line in "${hostnames[@]}"; do
  ssh ${line} `iostat -xtc 5 3 > /home/andra.lungu/cpu_stats/${line}` &

And then I parse the individual output in the cpu_stats folder with python
to form charts.
To my knowledge. Flink does not do something like that for you, at least
not yet :)

Bottom line is that you can use linux commands: iostat, dstat, htop, etc.
Hope this helps!

On Fri, Aug 7, 2015 at 5:07 PM, Stefanos Antaris <antaris.stefanos@gmail.com
> wrote:

> Hi to all.
> I am working an a research project using flink and i would like to extract
> the CPU + RAM resources consumed on each worker in order to include the
> stats in my paper. Can anyone  advice me on how could i extract them?
> Thanks in advance.
> Stefanos

View raw message