nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aldrin Piri <aldrinp...@gmail.com>
Subject Re: Dockerfile and Docker Hub Management
Date Tue, 26 Sep 2017 14:57:29 GMT
It seems like the separate release process is the one of preference.  I
have created a JIRA issue (https://issues.apache.org/jira/browse/NIFI-4425)
to capture some of the sentiment from this thread and to subsequently track
the efforts once we start executing upon them.  Feel free to add any
additional comments/thoughts to that ticket.

Thanks!
--aldrin

On Thu, Sep 21, 2017 at 1:44 PM, Adam Taft <adam@adamtaft.com> wrote:

> Aldrin,
>
> +1 to separate repository (bullet #2).  The basic premise that Docker
> releases should happen separate from the main distribution is spot on. I
> think a separate repository would help keep this separation.
>
> I tend to believe that the future of NiFi distributions will be via
> containerization. Making the Docker components somewhat a standalone
> initiative will help drive changes and innovation in this area.  I'd like
> to help see Docker become a first-class citizen for distributing, running
> and upgrading NiFi.
>
> Thanks,
>
> Adam
>
>
>
> On Thu, Sep 21, 2017 at 9:11 AM, Aldrin Piri <aldrinpiri@gmail.com> wrote:
>
> > Hey folks,
> >
> > ** This message turned out to be more detailed than anticipated.  In
> > summary, I propose consolidating Docker/container work with a separate
> > release process outside of the repository they are packaging.  Full
> > thoughts and background follow.  Any input would be appreciated!
> >
> > ---
> >
> > I've been working through providing some additional Docker capabilities
> for
> > the project and wanted to share some thoughts as well as possible plans
> to
> > help us be a bit more nimble and responsive to curating Dockerfiles and
> > their respective images on DockerHub.
> >
> > As a bit of context, we currently have the core NiFi project captured in
> > two Dockerfiles, one that is used in conjunction with a Maven plugin for
> > creating an image during the NiFi build (dockermaven), and another that
> is
> > used for building tagged releases on Docker Hub (dockerhub).  Both of
> these
> > artifacts, currently, reside in a nifi-docker project and are activated
> via
> > Maven profile, (-P docker).
> >
> > We've seen at times that this is a very coupled process and limits our
> > flexibility.  For instance, we had an ill-placed 'chown' which caused a
> > duplicating layer and causes our image to be doubly large.  While this
> has
> > been remedied, given current release processes, this is included with the
> > core nifi release and we have been unable to rectify that issue.
> >
> > Another issue is a very specific sequence of actions that needs to happen
> > with the current release for artifacts to be triggered correctly in
> Docker
> > Hub.  This can be seen in Section 6 of the release guide [1].  While
> there
> > are ways to rectify this if the timing isn't quite right and/or an error
> is
> > made, it can impose an additional burden on the INFRA team to facilitate
> > these requests as there currently is no capability for PMCs to manage
> their
> > Docker repositories directly.
> >
> > Ultimately, I think we should consider a separate release process for
> NiFi
> > Docker, and any associated efforts that may relate to those files.  In
> this
> > context, NiFi is encompassing of all projects/efforts in the project.
> > Additional efforts could comprise of examples of configuring NiFi to be
> > secured or clustered, receive data from MiNiFi instances, or using Docker
> > Compose or other orchestration frameworks. I have also noticed a number
> of
> > different areas across our work that are using Docker for integration
> > testing purposes.  With some planning and coordination, we could likely
> > consolidate many of these core resources/templates to allow us to reuse
> > them across efforts.
> >
> > I believe there are two approaches from an organizational standpoint that
> > let us execute on the separate release process effectively:
> >
> > 1.) Condense all Docker artifacts into the current NiFi repository [2].
> We
> > update our release for NiFi to exclude the Docker subtree to carry out
> our
> > normal release flow and provide the build/tooling for the Docker subtree
> to
> > be released on its own.
> >
> > 2.)  Establish a new git repository to handle Docker and any other
> > containerization efforts and migrate all existing resources into a file
> > structure that makes sense.
> >
> > My inclination is toward (2).
> >
> > Regardless of path chosen above, this frees us to handle updates and
> > improvements to container efforts when needed.  Any time we wanted to
> > release updates to Docker images, we could perform a separate release on
> > either the subtree of (1) or the repository of (2) and reference the
> > associated latest artifacts of NiFi.
> >
> > If you've made it this far, thanks for working through the wall of text
> and
> > would appreciate any thoughts or comments.
> >
> > [1] http://nifi.apache.org/release-guide.html
> > [2] https://git-wip-us.apache.org/repos/asf?p=nifi.git
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message