spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan (JIRA)" <>
Subject [jira] [Commented] (SPARK-4960) Interceptor pattern in receivers
Date Thu, 25 Dec 2014 04:40:13 GMT


Hari Shreedharan commented on SPARK-4960:

Awesome! One idea I had is to call it on the store(..) calls. This is a bit tricky though,
since some receivers call store(T) and some call store(ArrayBuffer[T]) and some call both
(like the Kafka one and soon the Flume one too).

> Interceptor pattern in receivers
> --------------------------------
>                 Key: SPARK-4960
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Streaming
>            Reporter: Tathagata Das
> Sometimes it is good to intercept a message received through a receiver and modify /
do something with the message before it is stored into Spark. This is often referred to as
the interceptor pattern. There should be general way to specify an interceptor function that
gets applied to all receivers. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message