spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lalwani, Jayesh" <>
Subject Re: Spark Structured streaming - Kakfa - slowness with query 0
Date Tue, 20 Oct 2020 18:11:52 GMT
Are you getting any output? Streaming jobs typically run forever, and keep processing data
as it comes in the input. If a streaming job is working well, it will typically generate output
at a certain cadence

From: KhajaAsmath Mohammed <>
Date: Tuesday, October 20, 2020 at 1:23 PM
To: "user @spark" <>
Subject: [EXTERNAL] Spark Structured streaming - Kakfa - slowness with query 0

CAUTION: This email originated from outside of the organization. Do not click links or open
attachments unless you can confirm the sender and know the content is safe.


I have started using spark structured streaming for reading data from kaka and the job is
very slow. Number of output rows keeps increasing in query 0 and the job is running forever.
any suggestions for this please?


View raw message