spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Somogyi (JIRA)" <>
Subject [jira] [Resolved] (SPARK-26213) Custom Receiver for Structured streaming
Date Thu, 06 Dec 2018 18:31:00 GMT


Gabor Somogyi resolved SPARK-26213.
    Resolution: Information Provided

> Custom Receiver for Structured streaming
> ----------------------------------------
>                 Key: SPARK-26213
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Structured Streaming
>    Affects Versions: 2.4.0
>            Reporter: Aarthi
>            Priority: Major
> Hi,
> I have implemented a Custom Receiver for a https/json data source by implementing the
Receievr abstract class as provided in the documentation here []
> This approach works on Spark streaming context  where the custom receiver class is passed
it receiverStream. However I would like the implement the same for Structured streaming as
each of the DStreams have a complex structure and need to be joined with each other based
on complex rules. ([])
>  Structured streaming uses the Spark Session object that takes in DataStreamReader which
is a final class. Please advice on how to implement the custom receiver for Strucutred Streaming. 
> Thanks,

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message