samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@gdfm.me>
Subject Re: Running SAMOA with Storm
Date Sun, 03 Apr 2016 07:51:16 GMT
I'm running the master version of SAMOA from github and Storm 0.9.6.

In my case they are VMs, 4GB of RAM per core.
I think Nicolas has a different setup.
No changes on the Storm configuration.

-- Gianmarco

On Sun, Apr 3, 2016 at 10:47 AM, Jean Paul Barddal <jpbarddal@gmail.com>
wrote:

> Hi Gianmarco and Nicolas,
>
> Yes, it does.
> Can you tell me which version of SAMOA and Storm you have running in your
> cluster?
> I will try to set up a new cluster from scratch to see if I manage to get
> it to work.
> Also, can you tell me if your clusters’ nodes are VMs? And you many RAM
> each has?
> Finally, do you have any changes to your garbage collection on storm, or
> everything is as default?
>
>
> Nicolas, I honestly don’t recall where I saw this example using com.yahoo
> classes.
> Should I use others instead?
>
> Cheers,
>
> Jean
>
>
> > On Apr 3, 2016, at 7:40 PM, Gianmarco De Francisci Morales <
> gdfm@apache.org> wrote:
> >
> > Hi Jean Paul,
> > I cannot reproduce the issue on my setup, the default bagging works fine
> on
> > my cluster.
> > Have you validated that your Storm cluster works for other applications?
> >
> > -- Gianmarco
> >
> > On Thu, Mar 24, 2016 at 1:40 PM, Nicolas Kourtellis <
> nkourtellis@gmail.com>
> > wrote:
> >
> >> Hi Jean,
> >>
> >> The example you mentioned running is using yahoo.labs code.
> >> Did you find these on the samoa - apache page? Can you point to the
> >> location?
> >>
> >> We may need to update those with the apache versions.
> >>
> >> In any case, this generator should now be on the master of apache repo.
> >>
> >> Thanks,
> >>
> >> Nicolas
> >>
> >> On Tue, Mar 8, 2016 at 2:54 AM, Jean Paul Barddal <jpbarddal@gmail.com>
> >> wrote:
> >>
> >>> Nope.
> >>> I’ve tested the default configuration provided in examples in the samoa
> >>> page, such as -s
> >>> (com.yahoo.labs.samoa.moa.streams.generators.RandomTreeGenerator -c 2
> -o
> >> 10
> >>> -u 10).
> >>>
> >>> Cheers,
> >>>
> >>> Jean
> >>>
> >>>> On Mar 8, 2016, at 12:18 AM, Gianmarco De Francisci Morales <
> >>> gdfm@apache.org> wrote:
> >>>>
> >>>> The Boosting is not very well tested, so I would expect some bugs
> >> there.
> >>>> Bagging is a bit more well tested.
> >>>> Are you using a dataset with a very large number of attributes?
> >>>>
> >>>> -- Gianmarco
> >>>>
> >>>> On Mon, Mar 7, 2016 at 2:55 AM, Jean Paul Barddal <
> jpbarddal@gmail.com
> >>>
> >>>> wrote:
> >>>>
> >>>>> Hello Gianmarco,
> >>>>>
> >>>>> I’ve tried both Bagging and Boosting.
> >>>>> They end up in a GC error pretty quickly.
> >>>>>
> >>>>> Jean
> >>>>>
> >>>>>
> >>>>>> On Mar 7, 2016, at 1:03 AM, Gianmarco De Francisci Morales <
> >>>>> gdfm@apache.org> wrote:
> >>>>>>
> >>>>>> Hi Jean Paul,
> >>>>>>
> >>>>>> Which ensemble are you running? Bagging?
> >>>>>> And what's the ensemble size?
> >>>>>>
> >>>>>> Nicolas, if I remember correctly we do not have acking for Storm
> >>> enabled
> >>>>> in
> >>>>>> the current version on master.
> >>>>>> Maybe this is why not all instances are executed?
> >>>>>> If so, I could port the acking we have on the VHT branch on
master.
> >>>>>>
> >>>>>> Cheers,
> >>>>>>
> >>>>>> -- Gianmarco
> >>>>>>
> >>>>>> On Thu, Feb 25, 2016 at 4:23 PM, Jean Paul Barddal <
> >>> jpbarddal@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hello again everyone,
> >>>>>>>
> >>>>>>> I managed to improve a little my situation.
> >>>>>>> Now, the problem only occurs with ensembles.
> >>>>>>> I am sending too screenshots of two issues I currently have.
> >>>>>>> If the amount of instances is to big (1 million or more),
then I
> ran
> >>>>> into
> >>>>>>> a GC problem.
> >>>>>>> If less, sometimes the evaluation node does not execute
all the
> >>>>> instances.
> >>>>>>> I’ve left the topology running over night and it was still
stuck.
> >>>>>>>
> >>>>>>> http://i67.tinypic.com/10cqnvp.png
> >>>>>>> http://i66.tinypic.com/f9mwrc.png
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Jean
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Feb 23, 2016, at 8:53 AM, Jean Paul Barddal <
> >> jpbarddal@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Dear Nicolas and Gianmarco,
> >>>>>>>>
> >>>>>>>> Thanks for the replies!
> >>>>>>>>
> >>>>>>>> The screenshot can be found at:
> http://i68.tinypic.com/25jdu9v.png
> >>>>>>>>
> >>>>>>>> Yes, I have tried a few examples from the guide. I’ve
tried VHT
> and
> >>>>>>> Bagging over a RandomTreeGenerator.
> >>>>>>>>
> >>>>>>>> Sometimes it works, but then I get a GC (garbage collector
error)
> >>> too.
> >>>>>>>>
> >>>>>>>> Gianmarco, I’m running Storm 0.9.5 and SAMOA 0.0.4.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>>
> >>>>>>>> Jean
> >>>>>>>>
> >>>>>>>>> On Feb 22, 2016, at 6:11 PM, Gianmarco De Francisci
Morales <
> >>>>>>> gdfm@apache.org> wrote:
> >>>>>>>>>
> >>>>>>>>> Also, which version of Storm are you using?
> >>>>>>>>>
> >>>>>>>>> -- Gianmarco
> >>>>>>>>>
> >>>>>>>>> On Mon, Feb 22, 2016 at 1:01 PM, Nicolas Kourtellis
<
> >>>>>>> nkourtellis@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Jean,
> >>>>>>>>>> Thanks for your interest in SAMOA!
> >>>>>>>>>> I dont think the attachment came through.
> >>>>>>>>>> Maybe you can post it in an online storage and
send a link?
> >>>>>>>>>>
> >>>>>>>>>> Regarding your question, have you tried the
basic examples from
> >> the
> >>>>>>> guide?
> >>>>>>>>>> E.g., the VHT in storm mode?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>>
> >>>>>>>>>> Nicolas
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Feb 22, 2016 at 5:21 PM, Jean Paul Barddal
<
> >>>>>>> jpbarddal@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hello guys,
> >>>>>>>>>>>
> >>>>>>>>>>> I am trying to set up a SAMOA environment
and I'm having some
> >>>>> issues.
> >>>>>>>>>>> I have the storm up and running and managed
to download,
> compile
> >>> and
> >>>>>>> run
> >>>>>>>>>>> SAMOA in local configuration.
> >>>>>>>>>>> Now I'm trying to run it in cluster mode.
> >>>>>>>>>>> As you can see from the attached screnshot,
the topology is up
> >> and
> >>>>>>>>>>> running, but the spouts do not send any
data to the bolts and
> so
> >>> on.
> >>>>>>>>>>>
> >>>>>>>>>>> Ideas?
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>>
> >>>>>>>>>>> Jean
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Nicolas Kourtellis
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Nicolas Kourtellis
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message