spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: Spark API - support for asynchronous calls - Reactive style [I]
Date Mon, 09 Dec 2013 17:31:33 GMT
Spark has already supported async jobs for awhile now --
https://github.com/apache/incubator-spark/pull/29, and they even work
correctly after https://github.com/apache/incubator-spark/pull/232

There are now implicit conversions from RDD to
AsyncRDDActions<https://github.com/apache/incubator-spark/blob/master/core/src/main/scala/org/apache/spark/rdd/AsyncRDDActions.scala>,
where async actions like countAsync are defined.


On Mon, Dec 9, 2013 at 5:46 AM, Deenar Toraskar <deenar.toraskar@db.com>wrote:

> Classification: For internal use only
> Hi developers
>
> Are there any plans to have Spark (and Shark) APIs that are asynchronous
> and non blocking? APIs that return Futures and Iteratee/Enumerators would
> be very useful to users building scalable apps using Spark, specially when
> combined with a fully asynchronous/non-blocking framework like Play!.
>
> Something along the lines of ReactiveMongo
>
> http://stephane.godbillon.com/2012/08/30/reactivemongo-for-scala-unleashing-mongodb-streaming-capabilities-for-realtime-web
>
>
> Deenar
>
> ---
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and delete this e-mail. Any
> unauthorized copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
> Please refer to http://www.db.com/en/content/eu_disclosures.htm for
> additional EU corporate and regulatory disclosures.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message