spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hendrik Dev <hendrikde...@gmail.com>
Subject Retrieve batch metadata via the spark monitoring api
Date Tue, 13 Feb 2018 14:20:37 GMT
I use Spark 2.2.1 with streaming and when i open the Spark Streaming
UI i can see input metadata for each of my batches. In my case i
stream from Kafka and in the metadata section i find useful
informations about my topic, partitions and offsets.

Assume the url for this batch looks like
http://localhost:4040/streaming/batch/?id=1566530100001

When i now query spark with the monitoring API there seems to be no
way to retrieve the above mentioned metadata.

Have i overlooked something or is there currently with spark 2.2.1
really no way to retrieve the input metadata via the REST API?

Thx
Hendrik

-- 
Hendrik Saly (salyh, hendrikdev22)
@hendrikdev22
PGP: 0x22D7F6EC

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message