spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Basavaraj <raj...@gmail.com>
Subject Re: Checking if cascading graph computation is possible in Spark
Date Fri, 05 Apr 2019 18:15:43 GMT
I have checked broadcast of accumulated values, but not satellite stateful
stabbing

But, I am not sure how that helps here

On Fri, 5 Apr 2019, 10:13 pm Jason Nerothin, <jasonnerothin@gmail.com>
wrote:

> Have you looked at Arbitrary Stateful Streaming and Broadcast Accumulators?
>
> On Fri, Apr 5, 2019 at 10:55 AM Basavaraj <rajiff@gmail.com> wrote:
>
>> Hi
>>
>> Have two questions
>>
>> #1
>> I am trying to process events in realtime, outcome of the processing has
>> to find a node in the GraphX and update that node as well (in case if any
>> anomaly or state change), If a node is updated, I have to update the
>> related nodes as well, want to know if GraphX can help in this by providing
>> some native support
>>
>> #2
>> I want to do the above as a event driven way, without using the batches
>> (i tried micro batches, but I realised that’s not what I want), i.e., for
>> each arriving event or as soon as a event message come my stream, not by
>> accumulating the event
>>
>> I humbly welcome any pointers, constructive criticism
>>
>> Regards
>> Basav
>> --------------------------------------------------------------------- To
>> unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
>
> --
> Thanks,
> Jason
>

Mime
View raw message