nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Taft <a...@adamtaft.com>
Subject Re: Dockerfile and Docker Hub Management
Date Thu, 21 Sep 2017 17:44:13 GMT
Aldrin,

+1 to separate repository (bullet #2).  The basic premise that Docker
releases should happen separate from the main distribution is spot on. I
think a separate repository would help keep this separation.

I tend to believe that the future of NiFi distributions will be via
containerization. Making the Docker components somewhat a standalone
initiative will help drive changes and innovation in this area.  I'd like
to help see Docker become a first-class citizen for distributing, running
and upgrading NiFi.

Thanks,

Adam



On Thu, Sep 21, 2017 at 9:11 AM, Aldrin Piri <aldrinpiri@gmail.com> wrote:

> Hey folks,
>
> ** This message turned out to be more detailed than anticipated.  In
> summary, I propose consolidating Docker/container work with a separate
> release process outside of the repository they are packaging.  Full
> thoughts and background follow.  Any input would be appreciated!
>
> ---
>
> I've been working through providing some additional Docker capabilities for
> the project and wanted to share some thoughts as well as possible plans to
> help us be a bit more nimble and responsive to curating Dockerfiles and
> their respective images on DockerHub.
>
> As a bit of context, we currently have the core NiFi project captured in
> two Dockerfiles, one that is used in conjunction with a Maven plugin for
> creating an image during the NiFi build (dockermaven), and another that is
> used for building tagged releases on Docker Hub (dockerhub).  Both of these
> artifacts, currently, reside in a nifi-docker project and are activated via
> Maven profile, (-P docker).
>
> We've seen at times that this is a very coupled process and limits our
> flexibility.  For instance, we had an ill-placed 'chown' which caused a
> duplicating layer and causes our image to be doubly large.  While this has
> been remedied, given current release processes, this is included with the
> core nifi release and we have been unable to rectify that issue.
>
> Another issue is a very specific sequence of actions that needs to happen
> with the current release for artifacts to be triggered correctly in Docker
> Hub.  This can be seen in Section 6 of the release guide [1].  While there
> are ways to rectify this if the timing isn't quite right and/or an error is
> made, it can impose an additional burden on the INFRA team to facilitate
> these requests as there currently is no capability for PMCs to manage their
> Docker repositories directly.
>
> Ultimately, I think we should consider a separate release process for NiFi
> Docker, and any associated efforts that may relate to those files.  In this
> context, NiFi is encompassing of all projects/efforts in the project.
> Additional efforts could comprise of examples of configuring NiFi to be
> secured or clustered, receive data from MiNiFi instances, or using Docker
> Compose or other orchestration frameworks. I have also noticed a number of
> different areas across our work that are using Docker for integration
> testing purposes.  With some planning and coordination, we could likely
> consolidate many of these core resources/templates to allow us to reuse
> them across efforts.
>
> I believe there are two approaches from an organizational standpoint that
> let us execute on the separate release process effectively:
>
> 1.) Condense all Docker artifacts into the current NiFi repository [2].  We
> update our release for NiFi to exclude the Docker subtree to carry out our
> normal release flow and provide the build/tooling for the Docker subtree to
> be released on its own.
>
> 2.)  Establish a new git repository to handle Docker and any other
> containerization efforts and migrate all existing resources into a file
> structure that makes sense.
>
> My inclination is toward (2).
>
> Regardless of path chosen above, this frees us to handle updates and
> improvements to container efforts when needed.  Any time we wanted to
> release updates to Docker images, we could perform a separate release on
> either the subtree of (1) or the repository of (2) and reference the
> associated latest artifacts of NiFi.
>
> If you've made it this far, thanks for working through the wall of text and
> would appreciate any thoughts or comments.
>
> [1] http://nifi.apache.org/release-guide.html
> [2] https://git-wip-us.apache.org/repos/asf?p=nifi.git
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message