spot-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chokha Palayamkottai <cho...@integralops.com>
Subject Re: Help on Spot installation (following Slack discussion with Nate)
Date Sat, 08 Apr 2017 19:19:02 GMT
Hi Dimitris,

Yes, I had the same issue. Unfortunately , I haven't spend a lot of time
on this since I have been busy with ingest. I was planning to use the
compiled .min.js files from the docker image and see if it helped.
Hopefully we can get some direction from the community.

-Chokha.

On 4/8/17 3:05 PM, Dimitris Papadopoulos wrote:
> Hi Chokha,
>
> It seems u were right, I can't find any .min.js inside the js/ dir. Mine
> looks like this:
>
> spotuser@master:~/spot-oa/ui/flow/js$ ls -l
> total 28
> drwxr-xr-x 2 root root 4096 Apr  4 10:40 components
> drwxr-xr-x 2 root root 4096 Apr  4 10:40 constants
> drwxr-xr-x 2 root root 4096 Apr  4 10:40 lib
> drwxr-xr-x 2 root root 4096 Apr  4 10:40 stores
> -rwxr-xr-x 1 root root 3229 Apr  4 10:40 storyboard.js
> -rwxr-xr-x 1 root root 3839 Apr  4 10:40 suspicious.js
> -rwxr-xr-x 1 root root 1719 Apr  4 10:40 threat-investigation.js
> spotuser@master:~/spot-oa/ui/flow/js$
>
> How did you fix this?
>
> Dimitris
>
> 2017-04-08 19:27 GMT+03:00 Chokha Palayamkottai <chokha@integralops.com>:
>
>> I ran into this issue couple weeks back and I believe it had to do with
>> SPOT UI build process. It wasn't generating the minified js files. Your
>> issue could probably be different but please verify if your build
>> process generated the *.min,js files.
>> https://github.com/apache/incubator-spot/tree/master/spot-oa/ui
>>
>> -Chokha.
>>
>> On 4/8/17 12:08 PM, Dimitris Papadopoulos wrote:
>>> Hi Diego,
>>>
>>> Thx for the quick response.
>>>
>>> Yes, I'm running it from oa-spot/. I just re-pasted the output of  ps
>> -ax |
>>> grep ipython at the end of this mail.
>>> Here are the screenshots (@Alan: thx for the info).
>>> Screenshot1: http://imgur.com/wR4g86M
>>> Screenshot2: http://imgur.com/MHA0sQQ
>>>
>>> The thing is that when I set up CDH on the 4 pseudo-nodes (master,
>>> s1,s2,s3), I realized that some of their assigned roles (I used the
>>> auto-assign option for the CDH roles)  didn't match the requirements for
>>> each Spot node, as described in the documentation.
>>> Thus, I used only 3 nodes as described below:
>>>
>>> *pseudo-node name | Spot node name:*
>>> master      | MASTER1, MASTER2, OA
>>> s3             | EDGE
>>> s2             |WORKER
>>>
>>> I know that the above setup wasn't going to provide any real
>> functionality,
>>> but then again I was testing the steps before moving to a real cluster.
>>>
>>>
>>> spotuser@master:~/spot-oa$ ps -ax | grep ipython
>>> 5953 ?        Sl     1:36 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_ia --
>>> cluster-id  --log-to-file --log-level=20
>>> 5962 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_ia --
>>> cluster-id  --log-to-file --log-level=20
>>> 5963 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_ia --
>>> cluster-id  --log-to-file --log-level=20
>>> 5966 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_ia --
>>> cluster-id  --log-to-file --log-level=20
>>> 5969 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_ia --
>>> cluster-id  --log-to-file --log-level=20
>>> 5974 ?        Sl     0:52 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus
>>> ter-id  --log-to-file --log-level=20
>>> 5976 ?        Sl     0:51 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus
>>> ter-id  --log-to-file --log-level=20
>>> 5981 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus
>>> ter-id  --log-to-file --log-level=20
>>> 5998 ?        Sl     0:50 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus
>>> ter-id  --log-to-file --log-level=20
>>> 10914 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
>> notebook
>>> --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>> 12417 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
>> notebook
>>> --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>> 19058 ?        Sl     1:34 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>> lt --cluster-id  --log-to-file --log-level=20
>>> 19066 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>> lt --cluster-id  --log-to-file --log-level=20
>>> 19067 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>> lt --cluster-id  --log-to-file --log-level=20
>>> 19070 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>> lt --cluster-id  --log-to-file --log-level=20
>>> 19074 ?        Sl     0:00 /usr/bin/python -m IPython.parallel.controller
>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>> lt --cluster-id  --log-to-file --log-level=20
>>> 19084 ?        Sl     0:52 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_default -
>>> -cluster-id  --log-to-file --log-level=20
>>> 19092 ?        Sl     0:51 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_default -
>>> -cluster-id  --log-to-file --log-level=20
>>> 19102 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_default -
>>> -cluster-id  --log-to-file --log-level=20
>>> 19118 ?        Sl     0:49 /usr/bin/python -m IPython.parallel.engine
>>> --profile-dir /home/spotuser/.ipython/profile_default -
>>> -cluster-id  --log-to-file --log-level=20
>>> 26655 lxc/tty1 R+     0:00 grep --color=auto ipython
>>> 26719 ?        R    156:55 /usr/bin/python /usr/local/bin/ipython
>> notebook
>>> --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>>
>>>
>>>
>>> 2017-04-08 18:03 GMT+03:00 Ross, Alan D <alan.d.ross@intel.com>:
>>>
>>>> No attachments here.  You can upload to imgur or somewhere else and then
>>>> post the link to the screenshot.
>>>>
>>>> -----Original Message-----
>>>> From: Diego (ChInO) [mailto:elchinillo@gmail.com]
>>>> Sent: Saturday, April 8, 2017 6:48 AM
>>>> To: dev@spot.incubator.apache.org
>>>> Subject: Re: Help on Spot installation (following Slack discussion with
>>>> Nate)
>>>>
>>>> Hi,
>>>>
>>>> I just want to make sure you are running ipython from the right place.
>>>> Could you please confirm you are running it from spot-oa/ dir?
>>>>
>>>> Btw, I don't see any screenshot :p
>>>>
>>>> On Fri, Apr 7, 2017, 11:05 PM Dimitris Papadopoulos <
>>>> dpapadopoulos91@gmail.com> wrote:
>>>>
>>>>> It appears i hadnt added myself to the dev mailing list.
>>>>>
>>>>> Hi again,
>>>>>
>>>>> First things first, let me recap our discussion so far:
>>>>>
>>>>> -We are trying to install Spot on a server with pseudo-distributed
>> nodes.
>>>>> -Each one of the nodes is actually an *lxc,* while the server runs on
>>>>> *Ubuntu 14.04*.
>>>>> -We started by installing CDH 10, following the cloudera site
>>>>> instructions and it seems to be working as intented.
>>>>> -We went on to install Spot, following the installation instructions
>>>>> as provided in the documentation webpage. Some parts (mainly where
>>>>> .conf files are mentioned) seem to be a bit vague for us, so we 've
>>>>> probably not configured the nodes right.
>>>>> -It seems that we haven't  managed to get Spot running (when we enter
>>>>> the ".../files/ui/flow/suspicious.html." we get an unresponsive page
>>>>> [Screenshot1] and we would like to fix that asap.
>>>>> -I have submitted two screenshots, one of the suspicious tab
>>>>> [Screenshot1]  and one of the ".../tree" view which lets me navigate
>>>>> normally through the files [Screenshot2]. At the end of the message is
>>>>> the result from the ps -ax | grep ipython
>>>>>
>>>>> We are probably asking too much effort from your side, but we could
>>>>> really benefit from a clean installation tutorial (video-blog-text
>>>>> format) that install all dependencies and sets all the configuration
>>>>> values for each one of the nodes. We could ofc provide a server for
>>>>> this purpose. Dempnstrating a Spot installation in a
>>>>> pseudo-distributed configuration could help us set it up on a large
>>>> distributed-computing infrastructure.
>>>>> Thank you so much for your time,
>>>>> Dimitris
>>>>>
>>>>> -------------------------------------------------------------------
>>>>> spotuser@master:~$ ps -ax | grep ipython
>>>>> 5953 ?        Sl     0:41 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 5962 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 5963 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 5966 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 5969 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 5974 ?        Sl     0:22 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
>>>>> --log-to-file --log-level=20
>>>>> 5976 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
>>>>> --log-to-file --log-level=20
>>>>> 5981 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
>>>>> --log-to-file --log-level=20
>>>>> 5998 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id
>>>>> --log-to-file --log-level=20
>>>>> 10914 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
>>>>> notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>>>> 12417 ?        S      0:00 /usr/bin/python /usr/local/bin/ipython
>>>>> notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>>>> 19058 ?        Sl     0:38 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>>>> lt --cluster-id  --log-to-file --log-level=20
>>>>> 19066 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>>>> lt --cluster-id  --log-to-file --log-level=20
>>>>> 19067 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>>>> lt --cluster-id  --log-to-file --log-level=20
>>>>> 19070 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>>>> lt --cluster-id  --log-to-file --log-level=20
>>>>> 19074 ?        Sl     0:00 /usr/bin/python -m
>> IPython.parallel.controller
>>>>> --profile-dir /home/spotuser/.ipython/profile_defau
>>>>> lt --cluster-id  --log-to-file --log-level=20
>>>>> 19084 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 19092 ?        Sl     0:21 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 19102 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 19118 ?        Sl     0:20 /usr/bin/python -m IPython.parallel.engine
>>>>> --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id
>>>>> --log-to-file --log-level=20
>>>>> 23679 lxc/tty1 S+     0:00 grep --color=auto ipython
>>>>> 26719 ?        S     65:15 /usr/bin/python /usr/local/bin/ipython
>>>>> notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0
>>>>> --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"]
>>>>> spotuser@master:~$
>>>>>
>>>>>
>>>>>
>>


Mime
View raw message