systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dusenberr...@gmail.com
Subject Re: [DISCUSS] SystemML with Spark 2.0 support and roadmap
Date Wed, 17 Aug 2016 23:51:34 GMT
+1

This sounds like a good plan to allow us to continue supporting the Spark 1.x line in the
short term, with a plan for moving to Spark 2.x support soon. 

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Aug 17, 2016, at 2:59 PM, Deron Eriksson <deroneriksson@gmail.com> wrote:
> 
> +1
> Continuing to support Spark 1.4/1.6 for now while setting a cutover date
> for 2.0 sounds like a great idea. This allows for the creation of a really
> solid release for 1.x, which greatly benefits SystemML users using Spark
> 1.x. It also gives these users a general date that they can use to plan
> migration to Spark 2.0 when that becomes the SystemML standard so that they
> can benefit from the latest improvements to the project.
> 
> Deron
> 
> 
>> On Wed, Aug 17, 2016 at 2:32 PM, Acs S <acs_s@yahoo.com.invalid> wrote:
>> 
>> Seems, mail is not retaining format. I am attaching same text through PDF
>> file.
>> If there is any other better option please let me know.
>> 
>> 
>> -Arvind
>> 
>> 
>> 
>> 
>> 
>> ----- Forwarded Message -----
>> *From:* Acs S <acs_s@yahoo.com.INVALID>
>> *To:* Dev <dev@systemml.incubator.apache.org>
>> *Sent:* Wednesday, August 17, 2016 2:18 PM
>> *Subject:* [DISCUSS] SystemML with Spark 2.0 support and roadmap
>> 
>> 
>> Spark 2.0 has released, we need to support SystemML on Spark 2.0 to be
>> uptodate with latest version of Spark. This brings us a challenge to
>> support our consumers until they move to Spark 2.0.Based on some
>> brainstorming, I can propose following options to keep SystemML being
>> supported on latest Spark version quickly.
>> 
>> Supporting SystemML on Spark 1.x We can continue to support SystemML on
>> Spark 1.x code base for short period of time by adding fixes and features
>> on main branch.  We will release SystemML with support to Spark 1.x next
>> version (0.11) around beginning of Oct 2016 (Lets target for Oct 1st 2016)
>> Supporting SystemML on Spark 2.0 (Preview code) For exploiters of Spark
>> 2.0, we can make SystemML on Spark 2.0 immediately based on branch created
>> on top of latest master branch code. Glen has some prototype code to
>> transform SystemML code to be compatible with Spark 2.0, he can merge his
>> code with new branch targeted to support SystemML on Spark 2.0 This would
>> be "Preview" version code, and we can update it on frequent basis (on
>> bi-monthy basis).  Supporting SystemML on Spark 2.0 We will have full
>> support of SystemML on Spark 2.0 before end of year 2016. We will formalize
>> release date by end of Sept 2016. At the same time we will discuss if we
>> can move support of SystemML on Spark 1.x to maintenance mode (Only
>> required bug fixes will be merged from main branch) or we need to support
>> both SystemML on Spark 2.0 and Spark 1.x for some additional time.
>> SystemML Roadmap 0.11 (on Spark 1.x) (Targeted to Oct 1st 2016) - Deep
>> Learning (Library of Network layers?) - Frame - New MLContext API        -
>> Python DSL integration (Preview) - Compressed Linear Algebra (Preview) -
>> Hydra R integration - New Algorithms (?)
>> 0.12 (Spark 2.0)  (Targeted to 4Q 2016) - GPU support (Local
>> mode/Distributed mode?)
>> - New Algorithms (?)
>> Please feel free to comment on support and roadmap points.
>> 
>> 
>> -Arvind
>> 
>> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message