spot-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dimitris Papadopoulos <dpapadopoulo...@gmail.com>
Subject Re: Help on Spot installation (following Slack discussion with Nate)
Date Sat, 08 Apr 2017 16:08:16 GMT
Hi Diego,

Thx for the quick response.

Yes, I'm running it from oa-spot/. I just re-pasted the output of  ps -ax |
grep ipython at the end of this mail.
Here are the screenshots (@Alan: thx for the info).
Screenshot1: http://imgur.com/wR4g86M
Screenshot2: http://imgur.com/MHA0sQQ

The thing is that when I set up CDH on the 4 pseudo-nodes (master,
s1,s2,s3), I realized that some of their assigned roles (I used the
auto-assign option for the CDH roles)  didn't match the requirements for
each Spot node, as described in the documentation.
Thus, I used only 3 nodes as described below:

*pseudo-node name | Spot node name:*
master      | MASTER1, MASTER2, OA
s3             | EDGE
s2             |WORKER

I know that the above setup wasn't going to provide any real functionality,
but then again I was testing the steps before moving to a real cluster.


spotuser@master:~/spot-oa$ ps -ax | grep ipython
5953 ?        Sl     1:36 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_ia --
cluster-id  --log-to-file --log-level=20
5962 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_ia --
cluster-id  --log-to-file --log-level=20
5963 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_ia --
cluster-id  --log-to-file --log-level=20
5966 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_ia --
cluster-id  --log-to-file --log-level=20
5969 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_ia --
cluster-id  --log-to-file --log-level=20
5974 ?        Sl     0:52 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_ia --clus
ter-id  --log-to-file --log-level=20
5976 ?        Sl     0:51 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_ia --clus
ter-id  --log-to-file --log-level=20
5981 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_ia --clus
ter-id  --log-to-file --log-level=20
5998 ?        Sl     0:50 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_ia --clus
ter-id  --log-to-file --log-level=20
10914 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython notebook
--no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
--no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
12417 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython notebook
--no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
--no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
19058 ?        Sl     1:34 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_defau
lt --cluster-id  --log-to-file --log-level=20
19066 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_defau
lt --cluster-id  --log-to-file --log-level=20
19067 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_defau
lt --cluster-id  --log-to-file --log-level=20
19070 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_defau
lt --cluster-id  --log-to-file --log-level=20
19074 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
--profile-dir /home/spotuser/.ipython/profile_defau
lt --cluster-id  --log-to-file --log-level=20
19084 ?        Sl     0:52 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_default -
-cluster-id  --log-to-file --log-level=20
19092 ?        Sl     0:51 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_default -
-cluster-id  --log-to-file --log-level=20
19102 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_default -
-cluster-id  --log-to-file --log-level=20
19118 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
--profile-dir /home/spotuser/.ipython/profile_default -
-cluster-id  --log-to-file --log-level=20
26655 lxc/tty1 R+     0:00 grep --color=auto ipython
26719 ?        R    156:55 /usr/bin/python /usr/local/bin/ipython notebook
--no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
--no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]



2017-04-08 18:03 GMT+03:00 Ross, Alan D <alan.d.ross@intel.com>:

> No attachments here.  You can upload to imgur or somewhere else and then
> post the link to the screenshot.
>
> -----Original Message-----
> From: Diego (ChInO) [mailto:elchinillo@gmail.com]
> Sent: Saturday, April 8, 2017 6:48 AM
> To: dev@spot.incubator.apache.org
> Subject: Re: Help on Spot installation (following Slack discussion with
> Nate)
>
> Hi,
>
> I just want to make sure you are running ipython from the right place.
> Could you please confirm you are running it from spot-oa/ dir?
>
> Btw, I don't see any screenshot :p
>
> On Fri, Apr 7, 2017, 11:05 PM Dimitris Papadopoulos <
> dpapadopoulos91@gmail.com> wrote:
>
> > It appears i hadnt added myself to the dev mailing list.
> >
> > Hi again,
> >
> > First things first, let me recap our discussion so far:
> >
> > -We are trying to install Spot on a server with pseudo-distributed nodes.
> > -Each one of the nodes is actually an *lxc,* while the server runs on
> > *Ubuntu 14.04*.
> > -We started by installing CDH 10, following the cloudera site
> > instructions and it seems to be working as intented.
> > -We went on to install Spot, following the installation instructions
> > as provided in the documentation webpage. Some parts (mainly where
> > .conf files are mentioned) seem to be a bit vague for us, so we 've
> > probably not configured the nodes right.
> > -It seems that we haven't  managed to get Spot running (when we enter
> > the ".../files/ui/flow/suspicious.html." we get an unresponsive page
> > [Screenshot1] and we would like to fix that asap.
> > -I have submitted two screenshots, one of the suspicious tab
> > [Screenshot1]  and one of the ".../tree" view which lets me navigate
> > normally through the files [Screenshot2]. At the end of the message is
> > the result from the ps -ax | grep ipython
> >
> > We are probably asking too much effort from your side, but we could
> > really benefit from a clean installation tutorial (video-blog-text
> > format) that install all dependencies and sets all the configuration
> > values for each one of the nodes. We could ofc provide a server for
> > this purpose. Dempnstrating a Spot installation in a
> > pseudo-distributed configuration could help us set it up on a large
> distributed-computing infrastructure.
> >
> > Thank you so much for your time,
> > Dimitris
> >
> > -------------------------------------------------------------------
> > spotuser@master:~$ ps -ax | grep ipython
> > 5953 ?        Sl     0:41 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
> > --log-to-file --log-level=20
> > 5962 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
> > --log-to-file --log-level=20
> > 5963 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
> > --log-to-file --log-level=20
> > 5966 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
> > --log-to-file --log-level=20
> > 5969 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
> > --log-to-file --log-level=20
> > 5974 ?        Sl     0:22 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
> > --log-to-file --log-level=20
> > 5976 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
> > --log-to-file --log-level=20
> > 5981 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
> > --log-to-file --log-level=20
> > 5998 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
> > --log-to-file --log-level=20
> > 10914 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
> > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
> > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
> > 12417 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
> > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
> > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
> > 19058 ?        Sl     0:38 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_defau
> > lt --cluster-id  --log-to-file --log-level=20
> > 19066 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_defau
> > lt --cluster-id  --log-to-file --log-level=20
> > 19067 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_defau
> > lt --cluster-id  --log-to-file --log-level=20
> > 19070 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_defau
> > lt --cluster-id  --log-to-file --log-level=20
> > 19074 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
> > --profile-dir /home/spotuser/.ipython/profile_defau
> > lt --cluster-id  --log-to-file --log-level=20
> > 19084 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
> > --log-to-file --log-level=20
> > 19092 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
> > --log-to-file --log-level=20
> > 19102 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
> > --log-to-file --log-level=20
> > 19118 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
> > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
> > --log-to-file --log-level=20
> > 23679 lxc/tty1 S+     0:00 grep --color=auto ipython
> > 26719 ?        S     65:15 /usr/bin/python /usr/local/bin/ipython
> > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
> > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
> > spotuser@master:~$
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message