From dev-return-1505-apmail-systemml-dev-archive=systemml.apache.org@systemml.incubator.apache.org Mon Mar 13 00:15:34 2017 Return-Path: X-Original-To: apmail-systemml-dev-archive@minotaur.apache.org Delivered-To: apmail-systemml-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C761A19EED for ; Mon, 13 Mar 2017 00:15:34 +0000 (UTC) Received: (qmail 9883 invoked by uid 500); 13 Mar 2017 00:15:34 -0000 Delivered-To: apmail-systemml-dev-archive@systemml.apache.org Received: (qmail 9827 invoked by uid 500); 13 Mar 2017 00:15:32 -0000 Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.incubator.apache.org Delivered-To: mailing list dev@systemml.incubator.apache.org Received: (qmail 9814 invoked by uid 99); 13 Mar 2017 00:15:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Mar 2017 00:15:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 28814182319 for ; Mon, 13 Mar 2017 00:15:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.648 X-Spam-Level: **** X-Spam-Status: No, score=4.648 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_REPLY=1, HTML_MESSAGE=2, KAM_LIVE=1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id XK1orbObmsi7 for ; Mon, 13 Mar 2017 00:15:28 +0000 (UTC) Received: from mail-ua0-f171.google.com (mail-ua0-f171.google.com [209.85.217.171]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D6D245F295 for ; Mon, 13 Mar 2017 00:15:27 +0000 (UTC) Received: by mail-ua0-f171.google.com with SMTP id u30so146104084uau.0 for ; Sun, 12 Mar 2017 17:15:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=9T32eyxGXEzWVJFtVLA32KYOLK5TGAy3lZ5+dHNAeiQ=; b=SxfzyCEp+2LiZdIg4eKEjJCfSaCYm+DZgNGyduopc89vTYBqN1oRzul76C4nvgaYd2 33seqHvIjbpO91nbH6yIyGvGBQ8J1wUKZDWVjbwrbNGgpMpWJ29SD/L9vkYs3FfrKaPs JwTebDWWIcHPR1EM7gOv511uJuVQuiwt4ff1RFsS7rJ+bTNF9LGLs+KHb8s3NwOYD2kI NWJwc66fctqiSgpUP6NSk5P6rCDA0QjI4nrbRhT53yMV4fOWl3Y/U4uYUdCTJ3W6aEPw lvg43Tp7SSbzwsgvNnuGrHi4ychg522HGYZ6OX7IpHQ4QAyQnY5YgGfXFyBQGLPU6dt1 sl+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=9T32eyxGXEzWVJFtVLA32KYOLK5TGAy3lZ5+dHNAeiQ=; b=Orc+kkNZJWzyz+ANwcc277ZbvGH1uC3fqu8OWLfax/ZCmrbs0llN/ZyXCiL8IUK0A/ Vm1LUNU1eU3yI5pewYR9KMyEvd9tOknVwp6SNg2tLynIorHpKnfms2BMY2CZpXO/sDTt 1Hla6Lxm6aLILH4PIuXcqn/s9kyaGGHjpxq7aA6XsUiF1z5SYRJ3+q/gEta1buBjnJp2 MMnCXk5Y7xJu2kk/D5yfnq1a0RHdenzDxYQe8RuzVifdZKcoTsb1suIE+kcrttdnTBAF iGNvPcVdTQm9POZK8aH415lPg6SAx10rG8K7+eOTqjQAHO0aDN8eXGMVcVaiaqkVDDu0 aE2w== X-Gm-Message-State: AMke39k0Iy4DWAP5Ccm3446o8zYtFVZ3epeFcLvhV4NGNc1MzTfDbVyd2XXXaQBZBzMBQbHcHrREra6gAFJLNQ== X-Received: by 10.176.90.208 with SMTP id x16mr15322813uae.158.1489364121082; Sun, 12 Mar 2017 17:15:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.38.119 with HTTP; Sun, 12 Mar 2017 17:15:20 -0700 (PDT) From: Matthias Boehm Date: Sun, 12 Mar 2017 17:15:20 -0700 Message-ID: Subject: Re: Release cadence To: dev@systemml.incubator.apache.org Content-Type: multipart/alternative; boundary=f403045f8b3860e272054a919bd3 --f403045f8b3860e272054a919bd3 Content-Type: text/plain; charset=UTF-8 ok great - as the majority is in favor of a 1.0 release and there is no veto, I think we came to an agreement here: our next release will be SystemML 1.0. Regards, Matthias On Tue, Mar 7, 2017 at 11:30 AM, wrote: > +1 for immediately starting work on SystemML 1.0 as our next release. > > At this point, the project and our users will benefit most from a thorough > cleanup, as it will make the project simpler to use and easier to > maintain. Simplicity will allow users and maintainers to regain focus on > ML research and products, which is a win for the entire community. We > should create a solid list of items that we, and the rest of the community, > want to address for the 1.0 release and make sure that they are indeed > completed. At the same time, we should ensure that we don't drag out the > release process. > > -Mike > > -- > > Mike Dusenberry > GitHub: github.com/dusenberrymw > LinkedIn: linkedin.com/in/mikedusenberry > > Sent from my iPhone. > > > > On Mar 6, 2017, at 10:14 AM, Luciano Resende > wrote: > > > > +1 for SystemML 1.0 as the next release. > > > > On Sat, Mar 4, 2017 at 10:23 AM, Deron Eriksson > > > wrote: > > > >> Personally I would like the next release to be 1.0. We have been an > >> incubator project since November 2015 and I believe that after over > 1,000 > >> commits since then that the project is about ready for a 1.0 release. > >> > >> I agree with Matthias that we need to make a decision regarding this > topic. > >> For new issues and fixed issues in JIRA, we need to be able to assign > the > >> correct version, or else someone potentially needs to go through and fix > >> the version numbers, as Glenn has been doing. Additionally, it would be > >> nice to do some of the 1.0 code updates (such as removing the old > >> MLContext) now rather than waiting additional months. Also I would like > to > >> be able to correctly identify our next version in the online > documentation. > >> > >> > > How about just make SystemML Next and change the release name when we do > > the release ? > > > > > > > >> Deron > >> > >> > >> On Sat, Mar 4, 2017 at 12:47 AM, Matthias Boehm > > >> wrote: > >> > >>> thanks Arvind for bringing some structure to the release process. I > >> think a > >>> fixed cadence of 2 months is useful as it makes upcoming releases more > >>> predictable for devs and users. > >>> > >>> However, we're discussing a major 1.0 release for a while now. I think > it > >>> would be useful to come to an agreement if we go for 1.0 in April or > not. > >>> There are some pending changes such as removing the old MLContext, > >> removing > >>> the file-based transform, isolating the matrix block library, and some > >>> language changes that should only be addresses in a major release as > they > >>> break backwards compatibility. Right now, we can't touch these changes > >>> without knowing the target release. > >>> > >>> Personally, I don't see a good reason why we should wait. Postponing > this > >>> major release just creates unnecessary overhead in maintaining these > old > >>> components that will be removed eventually. Since we cut RC for 0.13 on > >> Feb > >>> 20, I think having an RC around April 20 would be a good target for > this > >>> 1.0 release. > >>> > >>> > >>> Regards, > >>> Matthias > >>> > >>> > >>> On Fri, Mar 3, 2017 at 5:44 PM, Arvind Surve > >>> wrote: > >>> > >>>> Based on last couple of release cycles, we will continue with 2 months > >>>> release cycles.We will do first RC build by end of first week of > second > >>>> month. > >>>> We will plan on releasing next release by end of April 2017.We will > >> have > >>>> RC build on ~April 6th. -Arvind > >>>> Arvind Surve | Spark Technology Center | http://www.spark.tc/ > >>>> > >>>> From: Acs S > >>>> To: "dev@systemml.incubator.apache.org" >>>> apache.org> > >>>> Sent: Monday, January 9, 2017 11:41 AM > >>>> Subject: Re: Release cadence > >>>> > >>>> We need to release SystemML on more frequent basis to get community > >>>> engaged. It will provide us more feedback on functionality we > add.While > >>>> releasing SystemML on monthly basis is challenge due to longer phase > of > >>>> validation process we need to find a way to be quicker. > >>>> I can propose options to get closer to monthly release if acceptable. > >>>> Make every two releases available on monthly basis and third on two > >>> months > >>>> basis. This cycle will continue. > >>>> 1. Do minimal testing on two releases (minor releases) and release > them > >>> on > >>>> monthly basis. Performance testing is one of the major time consuming > >>>> activity especially for larger data size. We can limit testing only > >> upto > >>>> 80GB. We can do code freeze (other than fixes) at the end of third > week > >>> and > >>>> do verification on last week. If we find any issues we can still > >> release > >>>> the code with limitation documented unless issue breaks major > >>>> functionality.2. Do comprehensive testing on third release. This > will > >>>> include performance testing for all data size and every other testing > >> we > >>>> do. We can do code freeze (other than fixes) at the end of third week > >> and > >>>> start verification of code. All issues found will be addressed. > >> Exception > >>>> will be considered. > >>>> > >>>> Meantime we need to start automating testing pieces. > >>>> > >>>> Arvind SurveSpark Technology Centerhttp://www.spark.tc/ > >>>> > >>>> From: Berthold Reinwald > >>>> To: dev@systemml.incubator.apache.org > >>>> Sent: Saturday, January 7, 2017 1:35 AM > >>>> Subject: Re: Release cadence > >>>> > >>>> I think that a 2 month cycle would be a good compromise for > major/minor > >>>> releases. Fixpack release could be at a 1 month cycle. > >>>> > >>>> > >>>> Regards, > >>>> Berthold Reinwald > >>>> IBM Almaden Research Center > >>>> office: (408) 927 2208; T/L: 457 2208 > >>>> e-mail: reinwald@us.ibm.com > >>>> > >>>> > >>>> > >>>> From: Deron Eriksson > >>>> To: dev@systemml.incubator.apache.org > >>>> Date: 01/05/2017 02:14 PM > >>>> Subject: Re: Release cadence > >>>> > >>>> > >>>> > >>>> +1 for trying out a 1 month release cycle. > >>>> > >>>> However, I highly agree with Matthias that there is a lot of overhead > >>> with > >>>> releases, so it would be good if we can work to streamline/automate > the > >>>> process as much as possible. Also, it would be good to distribute the > >>>> tasks > >>>> around as much as possible. This can result in cross-training and help > >>>> avoid overburdening the same contributors each month. > >>>> > >>>> If the overhead slows us down too much, then we can go to a slower > >>> release > >>>> cycle. > >>>> > >>>> Deron > >>>> > >>>> > >>>> > >>>> > >>>>> On Thu, Jan 5, 2017 at 1:50 PM, wrote: > >>>>> > >>>>> +1 for adopting a 1 month release cycle. > >>>>> > >>>>> -- > >>>>> > >>>>> Mike Dusenberry > >>>>> GitHub: github.com/dusenberrymw > >>>>> LinkedIn: linkedin.com/in/mikedusenberry > >>>>> > >>>>> Sent from my iPhone. > >>>>> > >>>>> > >>>>>> On Jan 5, 2017, at 1:35 PM, Luciano Resende > >>>>> wrote: > >>>>>> > >>>>>> On Thu, Jan 5, 2017 at 6:05 AM, Matthias Boehm > >>>> > >>>>>> wrote: > >>>>>> > >>>>>>> In general, I like the idea of aiming for consistent release > >> cycles. > >>>>>>> However, every month is just too much, at least for me. There is a > >>>>>>> considerable overhead associated with each release for end-to-end > >>>>>>> performance tests, tests on different environments, code freeze > >> for > >>>> new > >>>>>>> features, etc. Hence, a too short release cycle would not be > >> "agile" > >>>> but > >>>>>>> would actually slow us down. From my perspective, a realistic > >>> release > >>>>>>> cadence would be 2-3 months, maybe a bit more for major releases. > >>>>>>> > >>>>>>> > >>>>>> 2-3 months of release cadence for an open source is probably a long > >>>>>> stretch, particular for a project that does not have very large set > >>> of > >>>>> 3rd > >>>>>> party dependencies. > >>>>>> > >>>>>> As for some of the overhead issues you mentioned, they are probably > >>>> easy > >>>>> to > >>>>>> workaround: > >>>>>> > >>>>>> - code-freeze timeframe can be resolved with branches > >>>>>> - end-to-end performance regressions can be avoided by better code > >>>>> review, > >>>>>> and if you were willing to go with 2-3 months without performing > >>> these > >>>>>> tests, we could perform them only for major releases, and > >> proactively > >>>>>> quickly build a minor release with the patch when a user report any > >>>>>> performance regression. > >>>>>> > >>>>>> > >>>>>> Anyway, I would really like to see SystemML more agile with regards > >>> to > >>>>> its > >>>>>> release process because, as I mentioned before, the release early, > >>>>> release > >>>>>> often mantra is good to increase community interest, generate more > >>>>> traffic > >>>>>> to the list as developers discuss the roadmap and release blockers, > >>>> and > >>>>>> also enable users to provide feedback sooner on the areas we are > >>>>> developing. > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Luciano Resende > >>>>>> http://twitter.com/lresende1975 > >>>>>> http://lresende.blogspot.com/ > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Deron Eriksson > >>>> Spark Technology Center > >>>> http://www.spark.tc/ > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> > >> > >> > >> > >> -- > >> Deron Eriksson > >> Spark Technology Center > >> http://www.spark.tc/ > >> > > > > > > > > -- > > Luciano Resende > > http://twitter.com/lresende1975 > > http://lresende.blogspot.com/ > --f403045f8b3860e272054a919bd3--