spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: queryable state & streaming
Date Fri, 08 Dec 2017 20:32:49 GMT
https://issues.apache.org/jira/browse/SPARK-16738

I don't believe anyone is working on it yet.  I think the most useful thing
is to start enumerating requirements and use cases and then we can talk
about how to build it.

On Fri, Dec 8, 2017 at 10:47 AM, Stavros Kontopoulos <
st.kontopoulos@gmail.com> wrote:

> Cool Burak do you have a pointer, should I take the initiative for a first
> design document or Databricks is working on it?
>
> Best,
> Stavros
>
> On Fri, Dec 8, 2017 at 8:40 PM, Burak Yavuz <brkyvz@gmail.com> wrote:
>
>> Hi Stavros,
>>
>> Queryable state is definitely on the roadmap! We will revamp the
>> StateStore API a bit, and a queryable StateStore is definitely one of the
>> things we are thinking about during that revamp.
>>
>> Best,
>> Burak
>>
>> On Dec 8, 2017 9:57 AM, "Stavros Kontopoulos" <st.kontopoulos@gmail.com>
>> wrote:
>>
>>> Just to re-phrase my question: Would query-able state make a viable
>>> SPIP?
>>>
>>> Regards,
>>> Stavros
>>>
>>> On Thu, Dec 7, 2017 at 1:34 PM, Stavros Kontopoulos <
>>> st.kontopoulos@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Maybe this has been discussed before. Given the fact that many
>>>> streaming apps out there use state extensively, could be a good idea to
>>>> make Spark expose streaming state with an external API like other
>>>> systems do (Kafka streams, Flink etc), in order to facilitate
>>>> interactive queries?
>>>>
>>>> Regards,
>>>> Stavros
>>>>
>>>
>>>
>

Mime
View raw message