madlib-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jingyi...@apache.org
Subject madlib git commit: Update dockerfile to use ubuntu 16.04
Date Mon, 05 Nov 2018 08:48:09 GMT
Repository: madlib
Updated Branches:
  refs/heads/master 30da14975 -> a61054c31


Update dockerfile to use ubuntu 16.04

This commit adds a new dockerfile to bake postgres 10.5 on ubuntu
16.04. Also updates docker_start.sh and README to pull the new docker image instead
of the old one (Postgres9.6 on Ubuntu 8.9).

Closes #332


Project: http://git-wip-us.apache.org/repos/asf/madlib/repo
Commit: http://git-wip-us.apache.org/repos/asf/madlib/commit/a61054c3
Tree: http://git-wip-us.apache.org/repos/asf/madlib/tree/a61054c3
Diff: http://git-wip-us.apache.org/repos/asf/madlib/diff/a61054c3

Branch: refs/heads/master
Commit: a61054c313426923617532200d1b2e6ca6ea3757
Parents: 30da149
Author: Jingyi Mei <jmei@pivotal.io>
Authored: Fri Oct 19 22:43:07 2018 +0800
Committer: Jingyi Mei <jmei@pivotal.io>
Committed: Mon Nov 5 11:50:24 2018 +0800

----------------------------------------------------------------------
 README.md                                       |   8 +-
 tool/docker/base/Dockerfile_ubuntu16_postgres10 | 108 +++++++++++++++++++
 tool/docker_start.sh                            |  10 +-
 3 files changed, 117 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/madlib/blob/a61054c3/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 4e218d5..d202722 100644
--- a/README.md
+++ b/README.md
@@ -20,7 +20,7 @@ also on the wiki.
 
 Development with Docker
 =======================
-We provide a Docker image with necessary dependencies required to compile and test MADlib
on PostgreSQL 9.6. You can view the dependency Docker file at ./tool/docker/base/Dockerfile_postgres_9_6.
The image is hosted on Docker Hub at madlib/postgres_9.6:latest. Later we will provide a similar
Docker image for Greenplum Database.
+We provide a Docker image with necessary dependencies required to compile and test MADlib
on PostgreSQL 10.5. You can view the dependency Docker file at ./tool/docker/base/Dockerfile_ubuntu16_postgres10.
The image is hosted on Docker Hub at madlib/postgres_10:latest. Later we will provide a similar
Docker image for Greenplum Database.
 
 We provide a script to quickly run this docker image at ./tool/docker_start.sh, which will
mount your local madlib directory, build MADlib and run install check on this Docker image.
At the end, it will `docker exec` as postgres user. Note that you have to run this script
from inside your madlib directory, and you can specify your docker CONTAINER_NAME (default
is madlib) and IMAGE_TAG (default is latest). Here is an example:
 
@@ -43,13 +43,13 @@ docker rm CONTAINER_NAME
 You can also manually run those commands to do the same thing:
 
 ```
-## 1) Pull down the `madlib/postgres_9.6:latest` image from docker hub:
-docker pull madlib/postgres_9.6:latest
+## 1) Pull down the `madlib/postgres_10:latest` image from docker hub:
+docker pull madlib/postgres_10:latest
 
 ## 2) Launch a container corresponding to the MADlib image, name it
 ##    madlib, mounting the source code folder to the container:
 docker run -d -it --name madlib \
-    -v (path to madlib directory):/madlib/ madlib/postgres_9.6
+    -v (path to madlib directory):/madlib/ madlib/postgres_10
 # where madlib is the directory where the MADlib source code resides.
 
 ################################# * WARNING * #################################

http://git-wip-us.apache.org/repos/asf/madlib/blob/a61054c3/tool/docker/base/Dockerfile_ubuntu16_postgres10
----------------------------------------------------------------------
diff --git a/tool/docker/base/Dockerfile_ubuntu16_postgres10 b/tool/docker/base/Dockerfile_ubuntu16_postgres10
new file mode 100644
index 0000000..cf7b4f2
--- /dev/null
+++ b/tool/docker/base/Dockerfile_ubuntu16_postgres10
@@ -0,0 +1,108 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM ubuntu:16.04
+
+### Get necessary libraries to add postgresql apt repository
+RUN apt-get update && apt-get install -y lsb-core software-properties-common wget
+
+### Add postgresql apt repository
+RUN add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -sc)-pgdg
main" && wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key
add - 
+
+### Have to update after getting new repository
+RUN apt-get update
+
+### Get postgres10 and postgres specific add-ons
+RUN apt-get install -y postgresql-10 \
+                       postgresql-client-10 \
+                       postgresql-plpython-10 \ 
+                       postgresql-server-dev-10 \
+                       libpq-dev \
+                       build-essential \
+                       openssl \
+                       libssl-dev \
+                       libboost-all-dev \
+                       m4 \
+                       vim \
+                       pgxnclient \
+                       flex \
+                       bison \
+                       graphviz
+
+### Reset pg_hba.conf file to allow no password prompt
+### Docker file doesn't support heardoc, like cat > /etc/postgresql/10/main/pg_hba.conf<<-EOF,
+### and this echo and \n\ are workaround to write the file
+RUN echo " \n\
+    # Database administrative login by Unix domain socket \n\
+    local   all             all                                     trust \n\
+
+    # TYPE  DATABASE        USER            ADDRESS                 METHOD \n\
+
+    # "local" is for Unix domain socket connections only \n\
+    local   all             all                                     trust \n\
+    # IPv4 local connections: \n\
+    host    all             all             127.0.0.1/32            trust \n\
+    # IPv6 local connections: \n\
+    host    all             all             ::1/128                 trust \n\
+" > /etc/postgresql/10/main/pg_hba.conf
+
+### We need to set nproc to unlimited to be able to run scripts as
+### the user 'postgres'. This is actually useful when we try to setup
+### and start a Postgres server.
+RUN echo " * soft nproc unlimited " > /etc/security/limits.d/postgres-limits.conf
+
+
+### Always start postgres server when login
+RUN echo "service postgresql start" >> ~/.bashrc
+
+### Build custom CMake with SSQL support
+RUN wget https://cmake.org/files/v3.6/cmake-3.6.1.tar.gz && \
+      tar -zxvf cmake-3.6.1.tar.gz && \
+      cd cmake-3.6.1 && \
+      sed -i 's/-DCMAKE_BOOTSTRAP=1/-DCMAKE_BOOTSTRAP=1 -DCMAKE_USE_OPENSSL=ON/g' bootstrap
&& \
+      ./configure &&  \
+      make -j2 && \
+      make install && \
+      cd ..
+
+### Install doxygen-1.8.13:
+RUN wget http://ftp.stack.nl/pub/users/dimitri/doxygen-1.8.13.src.tar.gz && \
+      tar xf doxygen-1.8.13.src.tar.gz && \
+      cd doxygen-1.8.13 && \
+      mkdir build && \
+      cd build && \
+      cmake -G "Unix Makefiles" .. && \
+      make && \
+      make install
+
+### Optional: install LaTex
+### uncomment the following 'RUN apt-get' line to bake LaTex into the image
+### Note: if you run the following line, please tag the image as
+### madlib/postgres_10:LaTex, and don't tag it as latest
+# RUN apt-get install -y texlive-full
+
+## To build an image from this docker file without LaTex, from madlib folder, run:
+## docker build -t madlib/postgres_10:latest -f tool/docker/base/Dockerfile_ubuntu16_postgres10
.
+## To push it to docker hub, run:
+## docker push madlib/postgres_10:latest
+
+## To build an image from this docker file with LaTex, from madlib folder, uncomment
+## line 88, and run:
+## docker build -t madlib/postgres_10:LaTex -f tool/docker/base/Dockerfile_ubuntu16_postgres10
.
+## To push it to docker hub, run:
+## docker push madlib/postgres_10:LaTex

http://git-wip-us.apache.org/repos/asf/madlib/blob/a61054c3/tool/docker_start.sh
----------------------------------------------------------------------
diff --git a/tool/docker_start.sh b/tool/docker_start.sh
index bc260c0..45762e1 100755
--- a/tool/docker_start.sh
+++ b/tool/docker_start.sh
@@ -18,7 +18,7 @@
 
 ###############################################################################
 #  This is a script that does the following:
-#  * Pull madlib/postgres_9.6:$IMAGE_TAG(default tag is `latest`) from docker;
+#  * Pull madlib/postgres_10:$IMAGE_TAG(default tag is `latest`) from docker;
 #  * Mount your local madlib directory to docker container;
 #  * Name your docker container as $CONTAINER_NAME (default name is madlib)
 #  * Build madlib from source; build dir is /madlib/build_docker which 
@@ -60,11 +60,11 @@ docker rm "${CONTAINER_NAME}"
 
 # Pull down the base docker images
 echo "Creating docker container"
-docker pull madlib/postgres_9.6:"${IMAGE_TAG}"
+docker pull madlib/postgres_10:"${IMAGE_TAG}"
 
 # Launch docker container with volume mounted from workdir
-docker run -d --name "${CONTAINER_NAME}" -v "${workdir}":/madlib \
-					madlib/postgres_9.6:"${IMAGE_TAG}" | tee build_docker_logs/docker_setup.log
+docker run -d -it --name "${CONTAINER_NAME}" -v "${workdir}":/madlib \
+					madlib/postgres_10:"${IMAGE_TAG}" | tee build_docker_logs/docker_setup.log
 
 ## This sleep is required since it takes a couple of seconds for the docker
 ## container to come up, which is required by the docker exec command that 
@@ -78,7 +78,7 @@ echo "---------- Building MADlib -----------"
 docker exec "${CONTAINER_NAME}" bash -c "rm -rf /madlib/build_docker; \
 							mkdir /madlib/build_docker; \
 							cd /madlib/build_docker; \
-							cmake ..; make; make doc; make install" \
+							cmake ..; make" \
 			| tee "${workdir}/build_docker_logs/madlib_compile.log"
 
 echo "---------- Installing and running install-check --------------------"


Mime
View raw message