spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Spark structured streaming
Date Tue, 08 Mar 2016 15:38:32 GMT
Hi Praveen,

I don't really know. I think TD or Michael should know as they
personally involved in the task (as far as I could figure it out from
the JIRA and the changes). Ping people on the JIRA so they notice your
question(s).

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Tue, Mar 8, 2016 at 12:32 PM, Praveen Devarao <praveendrl@in.ibm.com> wrote:
> Thanks Jacek for the pointer.
>
> Any idea which package can be used in .format(). The test cases seem to work
> out of the DefaultSource class defined within the DataFrameReaderWriterSuite
> [org.apache.spark.sql.streaming.test.DefaultSource]
>
> Thanking You
> ---------------------------------------------------------------------------------
> Praveen Devarao
> Spark Technology Centre
> IBM India Software Labs
> ---------------------------------------------------------------------------------
> "Courage doesn't always roar. Sometimes courage is the quiet voice at the
> end of the day saying I will try again"
>
>
>
> From:        Jacek Laskowski <jacek@japila.pl>
> To:        Praveen Devarao/India/IBM@IBMIN
> Cc:        user <user@spark.apache.org>, dev <dev@spark.apache.org>
> Date:        08/03/2016 04:17 pm
> Subject:        Re: Spark structured streaming
> ________________________________
>
>
>
> Hi Praveen,
>
> I've spent few hours on the changes related to streaming dataframes
> (included in the SPARK-8360) and concluded that it's currently only
> possible to read.stream(), but not write.stream() since there are no
> streaming Sinks yet.
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://medium.com/@jaceklaskowski/
> Mastering Apache Spark http://bit.ly/mastering-apache-spark
> Follow me at https://twitter.com/jaceklaskowski
>
>
> On Tue, Mar 8, 2016 at 10:38 AM, Praveen Devarao <praveendrl@in.ibm.com>
> wrote:
>> Hi,
>>
>>         I would like to get my hands on the structured streaming feature
>> coming out in Spark 2.0. I have tried looking around for code samples to
>> get
>> started but am not able to find any. Only few things I could look into is
>> the test cases that have been committed under the JIRA umbrella
>> https://issues.apache.org/jira/browse/SPARK-8360butthe test cases don't
>> lead to building a example code as they seem to be working out of internal
>> classes.
>>
>>         Could anyone point me to some resources or pointers in code that I
>> can start with to understand structured streaming from a consumability
>> angle.
>>
>> Thanking You
>>
>> ---------------------------------------------------------------------------------
>> Praveen Devarao
>> Spark Technology Centre
>> IBM India Software Labs
>>
>> ---------------------------------------------------------------------------------
>> "Courage doesn't always roar. Sometimes courage is the quiet voice at the
>> end of the day saying I will try again"
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message