nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <bimargul...@gmail.com>
Subject Re: [Discuss] preparing for release 0.0.1
Date Thu, 08 Jan 2015 23:32:32 GMT
If you don't want to release it you need to not deploy it. That is a pom
change unrelated to the staging repo.
On Jan 8, 2015 5:37 PM, "Adam Taft" <adam@adamtaft.com> wrote:

> Benson,
>
> Quite a bit of NIFI's artifacts are more like application bundles of other
> common libraries.  Most of the NIFI nars aren't really useful for
> application developers to bind to in their own code.  These nars are more
> part of the application and less useful as a shared common library.
>
> One of my concerns was accidentally releasing unneeded nars into maven
> central.  It sounds like the "purgatory" area solves this?  Can you speak
> to this a little more?
>
> There's been discussion on whether we care that application nars escape
> into maven central - I'm not trying to weigh in on that topic (though, I
> think you can read my thoughts between the lines).  I'm just asking for
> more clarity on the options when using the maven release plugin.  Purgatory
> might very well allow us to have our cake and eat it too.
>
> Thanks,
>
> Adam
>
>
> On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <bimargulies@gmail.com>
> wrote:
>
> > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <joe.witt@gmail.com> wrote:
> >
> > > Benson,
> > >
> > > Well I think we have all the groundwork laid as far as I know for the
> > > release plugin.  To be honest though (speaking for myself at least) I'm
> > not
> > > sure what the steps would be that the release plugin would do.  I'd
> love
> > it
> > > to be as you describe but am also concerned about how much we'd be
> > bugging
> > > you to figure that out.  I looked at what other projects seem to do and
> > it
> > > seemed shockingly manual.
> > >
> > > If you don't mind laying out the steps in more detail when you have
> time
> > > would love to learn from you on it.  I can also try toying around with
> > it a
> > > bit more off-line.
> > >
> >
> > I don't know what projects you're looking at, but it shouldn't look so
> > manual for any project that has a Maven top-level build.
> >
> > Here's the basic workflow of the maven-release-plugin:
> >
> > release:prepare:
> >
> > 1. edit poms to contain 'release version'.
> > 2. make a commit.
> > 3. make a tag.
> > 4. edit poms to contain 'next snapshot'
> > 5. make a commit.
> > 6. push the whole business (unless you turn that off).
> >
> > release:perform:
> >
> > 7. check out from tag in target/checkout (using extra git clone)
> > 8. build
> > 9. deploy results to target of deploymentManagement
> >
> > Now comes the next trick.
> >
> > Apache runs a copy of Sonatype Nexus, that serves as a 'staging
> > repository'.  So, when step 9 pushes things to the repository, it's
> pushing
> > them to a special 'purgatory' staging area. That area is available for
> > people to look at for testing, but does not propagate along to Maven
> > central.
> >
> > Given all this, then we add one ingredient: an additional maven module
> that
> > uses the maven-assembly-plugin to build a tarball that satisfies the
> > requirements of an Apache source release. This is not the same thing as
> > what the maven-source-plugin does at all. Since it will have
> > <attach>true</attach>, the resulting tarball swims upstream to the
> staging
> > repo with everything else.
> >
> > The vote email points people to that location, and supplies a sha1 for
> the
> > source ball. Voters download the source release from the staging repo, do
> > what they need to do, and vote. Three +1's later, and then you get to
> > publish.
> >
> > Does this help? The only manual steps are really copying to svn to push
> to
> > the Apache dist area. CXF and all the Maven plugins are examples.
> >
> >
> >
> >
> > >
> > > Thanks
> > > Joe
> > >
> > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <
> bimargulies@gmail.com>
> > > wrote:
> > >
> > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <joe.witt@gmail.com> wrote:
> > > >
> > > > > Benson,
> > > > >
> > > > > Yes that would be much appreciated.
> > > > >
> > > > > Here is a rough gamplan of what I *think* will be needed to do the
> > > > release:
> > > > >
> > > > > - Branch from develop against a ticket for the overall release
> > process
> > > > > - Update the version of all the artifacts and dependencies to
> > > > > 0.0.1-RC1-incubating or something like that
> > > > > - Create the tar.gz of the clean source tree.  Sign that and place
> > both
> > > > the
> > > > > signed file and the tar.gz into my people's dir (if i have one of
> > > those)
> > > > > - Notify folks that this is available so they can pull it and
> attempt
> > > to
> > > > > build from that and validate the signature
> > > > > - Once folks agree it is good to merge that to master
> > > > > - Create a tar.gz of that and a signed file.  Call a vote within
> the
> > > > team.
> > > > > If that is good call a vote within IPMC.
> > > > > - IF NO - fix whatever is wrong.
> > > > > - If all good then do a maven deploy of the binary artifacts,
> source
> > > > > bundles, and javadocs
> > > > > - Upload the tar.gz of source and the signed file to our dist dir.
> > And
> > > > > upload convenience binary build tar.gz and zip with sigs to our
> dist
> > > > > folder.  Then create links to these from our downloads page.
> > > > > - Update JIRA that the release is closed.
> > > > > - Wash/Rinse/Repeat
> > > > >
> > > > > <note this all is with the understanding that we've done all the
> > > > necessary
> > > > > review of dependencies, licenses, properly documented them, have
> our
> > > > > ECCN/encryption stuff properly filed>
> > > > >
> > > >
> > > > Sure, except that I think we can make the maven-release-plugin do a
> lot
> > > of
> > > > the work.
> > > >
> > > > The idea is that you run the release plugin, and it delivers all
> these
> > > > goodies to repository.apache.org. Then you call the vote. If the
> vote
> > > > passes, you (a) push the promote button to push to Maven Central, and
> > (b)
> > > > check the official source release tarball into the svn area for
> > > > distribution.
> > > >
> > > > I will take a whack at a PR for point (a).
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > Might be missing a detail or two but does that sound roughly on the
> > > right
> > > > > rack?
> > > > >
> > > > > I had though the maven release plugin would be useful but looks
> like
> > > > we'll
> > > > > just be able to use the versions plugin to update them and can do
> the
> > > > other
> > > > > simple steps manually.
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
> > > bimargulies@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Typically people set the maven assembly plugin for this and
> include
> > > it
> > > > in
> > > > > > the build. Would you like me to pitch this in?
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <madrob@cloudera.com>
> > > wrote:
> > > > > >
> > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <joe.witt@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > Josh
> > > > > > > >
> > > > > > > > Awesome.
> > > > > > > >
> > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
> > DISCLAIMER,
> > > > > > README.
> > > > > > > > All dependencies have been reviewed and checked for
ASLv2
> > > > > > compatibility.
> > > > > > > >
> > > > > > > > Will submit the infra ticket now for the dist/ path
for keys
> > > files
> > > > > and
> > > > > > > > such.
> > > > > > > >
> > > > > > > > My guess is that the release artifact should be a
tarball of
> > all
> > > > > > source.
> > > > > > > > Could we literally just package up a clean source
tree?
> Anyone
> > > > else
> > > > > > have
> > > > > > > > views on this?
> > > > > > > >
> > > > > > >
> > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag]
is
> a
> > > > > > perfectly
> > > > > > > reasonable thing to do.
> > > > > > >
> > > > > > > >
> > > > > > > > Ideally with this release we'd do it all properly
including
> > maven
> > > > > > > > artifacts, sources/javadocs, and so on.  The Maven
build does
> > > > already
> > > > > > > > operate now off a single command at the root to build
> > everything
> > > > > > > > (build-order is gone) and inherits from the apache
parent.
> > > > > > > >
> > > > > > > > Will need to incorporate RAT.
> > > > > > > >
> > > > > > > > Thanks for all that - definitely gives some stuff
to work on
> > and
> > > > look
> > > > > > > into.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Joe
> > > > > > > >
> > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
> > > josh.elser@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Regardless of what you call it, writing down
exactly what
> was
> > > > done
> > > > > to
> > > > > > > > make
> > > > > > > > > a RC is extremely important early on (I know
that I sure
> > can't
> > > > > > remember
> > > > > > > > > what I did last week, much less last release).
I'm not sure
> > if
> > > > > > release
> > > > > > > > > guide is formally defined somewhere, but enough
information
> > > that
> > > > > any
> > > > > > > > other
> > > > > > > > > committer can come by after "you get hit by a
train" and
> > make a
> > > > > > release
> > > > > > > > is
> > > > > > > > > extremely important. Using the CMS and making
a page on the
> > > > website
> > > > > > for
> > > > > > > > > this is extremely easy to do, IMO.
> > > > > > > > >
> > > > > > > > > Another concern for how to actually make a release
is the
> > type
> > > of
> > > > > > > > > packaging that is released. I not talking about
the
> > > source/binary
> > > > > > > release
> > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"?
Is
> > it a
> > > > > > > tarball?
> > > > > > > > > Are there jars in Maven? etc. A tarball is pretty
easy to
> > make,
> > > > and
> > > > > > > > likely
> > > > > > > > > the closest to what the build-order.sh script
already does
> > > (with
> > > > > > > Maven's
> > > > > > > > > help). I'm not sure if maven-released artifacts
are quite
> as
> > > > > > important
> > > > > > > at
> > > > > > > > > this point -- if it's not, punting on that will
help.
> If/when
> > > you
> > > > > get
> > > > > > > to
> > > > > > > > > that point, look into the apache parent pom[1].
It is
> > extremely
> > > > > well
> > > > > > > > > automated to use the existing infrastructure
to do it all
> for
> > > > you.
> > > > > > > > >
> > > > > > > > > Without looking at official documentation (which
is me
> being
> > > > lazy,
> > > > > > > > sorry),
> > > > > > > > > some other things that will need to be thought
about:
> > > > > > > > >
> > > > > > > > > Start with the releases page [2]
> > > > > > > > >
> > > > > > > > > * You *must* include a source artifact. It cannot
have
> > > binaries.
> > > > No
> > > > > > > Java
> > > > > > > > > class files, no jars. A user must be able to
download the
> > > source
> > > > > > > artifact
> > > > > > > > > and build it all themselves. Artifacts which
include
> binaries
> > > are
> > > > > for
> > > > > > > > > user-convenience only and can optionally be provided
as
> well.
> > > > > > > > >
> > > > > > > > > * LICENSE, NOTICE and README must all be included
in the
> top
> > > > level
> > > > > of
> > > > > > > the
> > > > > > > > > artifact. The NOTICE is the most painful and
must include
> any
> > > > > > required
> > > > > > > > > third-party notices. [3]
> > > > > > > > >
> > > > > > > > > * You will need to audit your dependencies to
make sure
> that
> > > they
> > > > > are
> > > > > > > > > ASLv2 compatible [4]
> > > > > > > > >
> > > > > > > > > * Releases must be cryptographically signed (PGP)
[5]. Your
> > > > public
> > > > > > key
> > > > > > > > > should be published to known sites (e.g. pgp.mit.edu)
and
> > must
> > > > be
> > > > > > > listed
> > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist,
probably
> > > needs
> > > > > an
> > > > > > > > infra
> > > > > > > > > ticket to create your dist/ directory?).
> > > > > > > > >
> > > > > > > > > * Verify that every source file contain the proper
ASL
> > header.
> > > > The
> > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`)
is a
> wonderful
> > > > tool
> > > > > > that
> > > > > > > > can
> > > > > > > > > (and should, IMO) be integrated into your build
process to
> > > > prevent
> > > > > > > people
> > > > > > > > > from accidentally committing new files between
releases
> that
> > do
> > > > not
> > > > > > > have
> > > > > > > > > the correct header.
> > > > > > > > >
> > > > > > > > > And suddenly, this is really long :). Short answer:
decide
> > what
> > > > the
> > > > > > > > > release should look like (just a tarball?), start
vetting
> > your
> > > > > source
> > > > > > > > code
> > > > > > > > > for license headers, and looking into NiFi's
dependencies
> and
> > > > their
> > > > > > > > > licenses.
> > > > > > > > >
> > > > > > > > > [1] http://maven.apache.org/pom/asf/
> > > > > > > > > [2] http://www.apache.org/dev/release.html
> > > > > > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > > > > > [7] http://creadur.apache.org/rat/
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Tony Kurc wrote:
> > > > > > > > >
> > > > > > > > >> I read the guide Joe linked and a lot of
the sticky parts
> > are
> > > > > marked
> > > > > > > > >> "TODO"
> > > > > > > > >> and it looks like a work in progress
> > > > > > > > >>
> > > > > > > > >> "Podlings can short circuit this process
by starting out
> > with
> > > > > > written
> > > > > > > > >> release documentation. It is strongly recommended
that
> > > Podlings
> > > > > > invest
> > > > > > > > >> time
> > > > > > > > >> looking at the best practices recommended
in this
> document.
> > By
> > > > > > > selection
> > > > > > > > >> and modification, sections of this document
can be used to
> > > > quickly
> > > > > > and
> > > > > > > > >> easily bootstrap a release guide. "
> > > > > > > > >>
> > > > > > > > >> Is step one putting together a release guide?
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<
> joe.witt@gmail.com
> > >
> > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >>  Hello
> > > > > > > > >>>
> > > > > > > > >>> Just wanted to stir this one up a bit.
 Looks like all
> > > tickets
> > > > > > pegged
> > > > > > > > as
> > > > > > > > >>> 0.0.1 are resolved (2 remain as of now
but seem likely to
> > be
> > > > > > resolved
> > > > > > > > >>> shortly based on their comments).  So
working through the
> > > > release
> > > > > > > steps
> > > > > > > > >>> available on apache.org and via the link
Brock sent.
> > > > > > > > >>>
> > > > > > > > >>> Anyone interested in this part of the
process or who has
> > > advice
> > > > > to
> > > > > > > help
> > > > > > > > >>> us
> > > > > > > > >>> avoid landmines we're happy to hear it.
> > > > > > > > >>>
> > > > > > > > >>> Thanks
> > > > > > > > >>> Joe
> > > > > > > > >>>
> > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe
Witt<
> > joe.witt@gmail.com
> > > >
> > > > > > > wrote:
> > > > > > > > >>>
> > > > > > > > >>>  Thanks Brock this is very helpful.
> > > > > > > > >>>>
> > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM,
Brock Noland<
> > > > > brock@cloudera.com>
> > > > > > > > >>>>
> > > > > > > > >>> wrote:
> > > > > > > > >>>
> > > > > > > > >>>> Hi,
> > > > > > > > >>>>>
> > > > > > > > >>>>> This is a decent guide which
can be copied:
> > > > > > > > >>>>>
> > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > > > > > >>>>>
> > > > > > > > >>>>> Brock
> > > > > > > > >>>>>
> > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17
AM, Joe Witt<
> > > > joe.witt@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >>>>>
> > > > > > > > >>>>>> Folks,
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Looking at the tickets that
remain which are presently
> > > tied
> > > > to
> > > > > > > 0.0.1
> > > > > > > > >>>>>>
> > > > > > > > >>>>> we're
> > > > > > > > >>>>>
> > > > > > > > >>>>>> probably 1-2 weeks out from
this initial release.  Can
> > you
> > > > > > provide
> > > > > > > > >>>>>>
> > > > > > > > >>>>> some
> > > > > > > > >>>
> > > > > > > > >>>> pointers/references or pointers on
how to get this ball
> > > > rolling
> > > > > > and
> > > > > > > > >>>>>>
> > > > > > > > >>>>> any
> > > > > > > > >>>
> > > > > > > > >>>> rocks we must move before going down
this path?
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > http://incubator.apache.org/guides/releasemanagement.html
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> That link seems really helpful
but has a lot of TODOs
> > for
> > > > > areas
> > > > > > > > which
> > > > > > > > >>>>>>
> > > > > > > > >>>>> need
> > > > > > > > >>>>>
> > > > > > > > >>>>>> more explanation.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Thanks
> > > > > > > > >>>>>> Joe
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message