systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Berthold Reinwald" <>
Subject [Discuss} SystemML Roadmap page
Date Fri, 23 Sep 2016 19:10:46 GMT
In the spirit of other Apache projects, we should publish a Roadmap page. 
The page should be clear on the immediate timeline, point to some future 
projects, and also summarize the past to demonstrate continuum. It is not 
a replacement for jiras or release notes, but just a single place for 
people to go to and see what happened in the past and what will happen 
going forward.

SystemML Roadmap

SystemML Release Timeline

Oct. 2016: SystemML 0.11.0 on Spark 1.x

Nov. 2016: SystemML 0.11.1 on Spark 1.x/2.x 

Dec. 2016: SystemML 1.0 on Spark 1.x/2.x

Next SystemML 0.11.x 
- Features
  -- SystemML frames
  -- New MLContext API
  -- Transform functions based on SystemML frames
- Bug fixes
- Experimental Features / algorithms
  -- New built-in functions for deep learning (convolution and pooling)
  -- Deep learning library (DML bodied functions)
  -- Python DSL integration
  -- GPU support
  -- Compressed Linear Algebra
  -- New Algorithms 
     --- Lasso
     --- kNN
     --- Lanczos
     --- PPCA
     --- Deep Learning: CNN (Lenet), RBM

Planned for future SystemML 1.0
- Rigorous performance and scalability testing (bug fixes)
- Remove deprecated APIs
- Remove deprecated functions

Planned for future Releases
- Completion of prior experimental features
- New algorithms: Non-linear SVMs, solvers, decomposition, inversion, etc.
- DSLs (e.g. Scala, Python) and common DSL architecture
- R interfaces: R DSL and R wrappers
- Native Zeppelin Notebook support
- Code generation
- Sum product optimizations
- Tree-based data structures
- Global dataflow optimizations

Prior Releases

SystemML 0.10.0-incubating (released in June, 2016) (link to release notes 
- Different types of Spark Matrix Blocks: MCSR, CSR, COO
- SystemML Frame support in JMLC/CP
- Initial Deep Learning support
- API/Scripts: parser error handling, SystemML configuration handling, 
include algorithms in SystemML jar, print matrix
- New fused operator: wdivmm with variations
- Performance Features: cache-conscious operations, more multithreaded 
operations, new simplications rewrites
- New Algorithms: kNN
- Documentation: javadocs, Jupyter/Zeppeling notebook examples

SystemML 0.9.0-incubating (released in Jan. 2016) (link to release notes (
- Improvements to MLContext and MLPipeline wrappers
- New converter utilities for RDDs and DataFrames)
- New Optimizations for Spark Backend, e.g. eager RDD caching and 
repartitioning, RDD checkpointing, on-demand creation of SparkContext
- New Runtime Operators for mmult, multihreaded readers and operators.
- New Algoriths: ALS, Cubic Splines
- Online documentation

Berthold Reinwald
IBM Almaden Research Center
office: (408) 927 2208; T/L: 457 2208

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message