systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deron Eriksson <deroneriks...@gmail.com>
Subject Re: Fw: [DISCUSS] SystemML with Spark 2.0 support and roadmap
Date Wed, 17 Aug 2016 21:59:34 GMT
+1
Continuing to support Spark 1.4/1.6 for now while setting a cutover date
for 2.0 sounds like a great idea. This allows for the creation of a really
solid release for 1.x, which greatly benefits SystemML users using Spark
1.x. It also gives these users a general date that they can use to plan
migration to Spark 2.0 when that becomes the SystemML standard so that they
can benefit from the latest improvements to the project.

Deron


On Wed, Aug 17, 2016 at 2:32 PM, Acs S <acs_s@yahoo.com.invalid> wrote:

> Seems, mail is not retaining format. I am attaching same text through PDF
> file.
> If there is any other better option please let me know.
>
>
> -Arvind
>
>
>
>
>
> ----- Forwarded Message -----
> *From:* Acs S <acs_s@yahoo.com.INVALID>
> *To:* Dev <dev@systemml.incubator.apache.org>
> *Sent:* Wednesday, August 17, 2016 2:18 PM
> *Subject:* [DISCUSS] SystemML with Spark 2.0 support and roadmap
>
>
> Spark 2.0 has released, we need to support SystemML on Spark 2.0 to be
> uptodate with latest version of Spark. This brings us a challenge to
> support our consumers until they move to Spark 2.0.Based on some
> brainstorming, I can propose following options to keep SystemML being
> supported on latest Spark version quickly.
>
> Supporting SystemML on Spark 1.x We can continue to support SystemML on
> Spark 1.x code base for short period of time by adding fixes and features
> on main branch.  We will release SystemML with support to Spark 1.x next
> version (0.11) around beginning of Oct 2016 (Lets target for Oct 1st 2016)
> Supporting SystemML on Spark 2.0 (Preview code) For exploiters of Spark
> 2.0, we can make SystemML on Spark 2.0 immediately based on branch created
> on top of latest master branch code. Glen has some prototype code to
> transform SystemML code to be compatible with Spark 2.0, he can merge his
> code with new branch targeted to support SystemML on Spark 2.0 This would
> be "Preview" version code, and we can update it on frequent basis (on
> bi-monthy basis).  Supporting SystemML on Spark 2.0 We will have full
> support of SystemML on Spark 2.0 before end of year 2016. We will formalize
> release date by end of Sept 2016. At the same time we will discuss if we
> can move support of SystemML on Spark 1.x to maintenance mode (Only
> required bug fixes will be merged from main branch) or we need to support
> both SystemML on Spark 2.0 and Spark 1.x for some additional time.
> SystemML Roadmap 0.11 (on Spark 1.x) (Targeted to Oct 1st 2016) - Deep
> Learning (Library of Network layers?) - Frame - New MLContext API        -
> Python DSL integration (Preview) - Compressed Linear Algebra (Preview) -
> Hydra R integration - New Algorithms (?)
> 0.12 (Spark 2.0)  (Targeted to 4Q 2016) - GPU support (Local
> mode/Distributed mode?)
> - New Algorithms (?)
> Please feel free to comment on support and roadmap points.
>
>
> -Arvind
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message