hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marton, Elek" <...@anzix.net>
Subject Re: [DISCUSS] official docker image(s) for hadoop
Date Fri, 22 Sep 2017 09:23:11 GMT
Thanks all the feedbacks.

I created an issue:

Let's continue the discussion there.


On 09/08/2017 02:45 PM, Marton, Elek wrote:
> TL;DR: I propose to create official hadoop images and upload them to the 
> dockerhub.
> GOAL/SCOPE: I would like improve the existing documentation with 
> easy-to-use docker based recipes to start hadoop clusters with various 
> configuration.
> The images also could be used to test experimental features. For example 
> ozone could be tested easily with these compose file and configuration:
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> Or even the configuration could be included in the compose file:
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml 
> I would like to create separated example compose files for federation, 
> ha, metrics usage, etc. to make it easier to try out and understand the 
> features.
> CONTEXT: There is an existing Jira 
> https://issues.apache.org/jira/browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images 
> (multiple types, in a flexible way). If no objections, I will create a 
> separated issue to create simplified docker images for rapid prototyping 
> and investigating new features. And register the branch to the dockerhub 
> to create the images automatically.
> MY BACKGROUND: I am working with docker based hadoop/spark clusters 
> quite a while and run them succesfully in different environments 
> (kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is 
> available from here: https://github.com/flokkr but they could handle 
> more complex use cases (eg. instrumenting java processes with btrace, or 
> read/reload configuration from consul).
>   And IMHO in the official hadoop documentation it’s better to suggest 
> to use official apache docker images and not external ones (which could 
> be changed).
> Please let me know if you have any comments.
> Marton
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message