samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean Paul Barddal <jpbard...@gmail.com>
Subject Re: Running SAMOA with Storm
Date Sun, 03 Apr 2016 07:47:16 GMT
Hi Gianmarco and Nicolas,

Yes, it does.
Can you tell me which version of SAMOA and Storm you have running in your cluster?
I will try to set up a new cluster from scratch to see if I manage to get it to work.
Also, can you tell me if your clusters’ nodes are VMs? And you many RAM each has?
Finally, do you have any changes to your garbage collection on storm, or everything is as
default?


Nicolas, I honestly don’t recall where I saw this example using com.yahoo classes.
Should I use others instead?

Cheers,

Jean


> On Apr 3, 2016, at 7:40 PM, Gianmarco De Francisci Morales <gdfm@apache.org> wrote:
> 
> Hi Jean Paul,
> I cannot reproduce the issue on my setup, the default bagging works fine on
> my cluster.
> Have you validated that your Storm cluster works for other applications?
> 
> -- Gianmarco
> 
> On Thu, Mar 24, 2016 at 1:40 PM, Nicolas Kourtellis <nkourtellis@gmail.com>
> wrote:
> 
>> Hi Jean,
>> 
>> The example you mentioned running is using yahoo.labs code.
>> Did you find these on the samoa - apache page? Can you point to the
>> location?
>> 
>> We may need to update those with the apache versions.
>> 
>> In any case, this generator should now be on the master of apache repo.
>> 
>> Thanks,
>> 
>> Nicolas
>> 
>> On Tue, Mar 8, 2016 at 2:54 AM, Jean Paul Barddal <jpbarddal@gmail.com>
>> wrote:
>> 
>>> Nope.
>>> I’ve tested the default configuration provided in examples in the samoa
>>> page, such as -s
>>> (com.yahoo.labs.samoa.moa.streams.generators.RandomTreeGenerator -c 2 -o
>> 10
>>> -u 10).
>>> 
>>> Cheers,
>>> 
>>> Jean
>>> 
>>>> On Mar 8, 2016, at 12:18 AM, Gianmarco De Francisci Morales <
>>> gdfm@apache.org> wrote:
>>>> 
>>>> The Boosting is not very well tested, so I would expect some bugs
>> there.
>>>> Bagging is a bit more well tested.
>>>> Are you using a dataset with a very large number of attributes?
>>>> 
>>>> -- Gianmarco
>>>> 
>>>> On Mon, Mar 7, 2016 at 2:55 AM, Jean Paul Barddal <jpbarddal@gmail.com
>>> 
>>>> wrote:
>>>> 
>>>>> Hello Gianmarco,
>>>>> 
>>>>> I’ve tried both Bagging and Boosting.
>>>>> They end up in a GC error pretty quickly.
>>>>> 
>>>>> Jean
>>>>> 
>>>>> 
>>>>>> On Mar 7, 2016, at 1:03 AM, Gianmarco De Francisci Morales <
>>>>> gdfm@apache.org> wrote:
>>>>>> 
>>>>>> Hi Jean Paul,
>>>>>> 
>>>>>> Which ensemble are you running? Bagging?
>>>>>> And what's the ensemble size?
>>>>>> 
>>>>>> Nicolas, if I remember correctly we do not have acking for Storm
>>> enabled
>>>>> in
>>>>>> the current version on master.
>>>>>> Maybe this is why not all instances are executed?
>>>>>> If so, I could port the acking we have on the VHT branch on master.
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> -- Gianmarco
>>>>>> 
>>>>>> On Thu, Feb 25, 2016 at 4:23 PM, Jean Paul Barddal <
>>> jpbarddal@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hello again everyone,
>>>>>>> 
>>>>>>> I managed to improve a little my situation.
>>>>>>> Now, the problem only occurs with ensembles.
>>>>>>> I am sending too screenshots of two issues I currently have.
>>>>>>> If the amount of instances is to big (1 million or more), then
I ran
>>>>> into
>>>>>>> a GC problem.
>>>>>>> If less, sometimes the evaluation node does not execute all the
>>>>> instances.
>>>>>>> I’ve left the topology running over night and it was still
stuck.
>>>>>>> 
>>>>>>> http://i67.tinypic.com/10cqnvp.png
>>>>>>> http://i66.tinypic.com/f9mwrc.png
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> Jean
>>>>>>> 
>>>>>>> 
>>>>>>>> On Feb 23, 2016, at 8:53 AM, Jean Paul Barddal <
>> jpbarddal@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Dear Nicolas and Gianmarco,
>>>>>>>> 
>>>>>>>> Thanks for the replies!
>>>>>>>> 
>>>>>>>> The screenshot can be found at: http://i68.tinypic.com/25jdu9v.png
>>>>>>>> 
>>>>>>>> Yes, I have tried a few examples from the guide. I’ve tried
VHT and
>>>>>>> Bagging over a RandomTreeGenerator.
>>>>>>>> 
>>>>>>>> Sometimes it works, but then I get a GC (garbage collector
error)
>>> too.
>>>>>>>> 
>>>>>>>> Gianmarco, I’m running Storm 0.9.5 and SAMOA 0.0.4.
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> 
>>>>>>>> Jean
>>>>>>>> 
>>>>>>>>> On Feb 22, 2016, at 6:11 PM, Gianmarco De Francisci Morales
<
>>>>>>> gdfm@apache.org> wrote:
>>>>>>>>> 
>>>>>>>>> Also, which version of Storm are you using?
>>>>>>>>> 
>>>>>>>>> -- Gianmarco
>>>>>>>>> 
>>>>>>>>> On Mon, Feb 22, 2016 at 1:01 PM, Nicolas Kourtellis <
>>>>>>> nkourtellis@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi Jean,
>>>>>>>>>> Thanks for your interest in SAMOA!
>>>>>>>>>> I dont think the attachment came through.
>>>>>>>>>> Maybe you can post it in an online storage and send
a link?
>>>>>>>>>> 
>>>>>>>>>> Regarding your question, have you tried the basic
examples from
>> the
>>>>>>> guide?
>>>>>>>>>> E.g., the VHT in storm mode?
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> 
>>>>>>>>>> Nicolas
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Mon, Feb 22, 2016 at 5:21 PM, Jean Paul Barddal
<
>>>>>>> jpbarddal@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello guys,
>>>>>>>>>>> 
>>>>>>>>>>> I am trying to set up a SAMOA environment and
I'm having some
>>>>> issues.
>>>>>>>>>>> I have the storm up and running and managed to
download, compile
>>> and
>>>>>>> run
>>>>>>>>>>> SAMOA in local configuration.
>>>>>>>>>>> Now I'm trying to run it in cluster mode.
>>>>>>>>>>> As you can see from the attached screnshot, the
topology is up
>> and
>>>>>>>>>>> running, but the spouts do not send any data
to the bolts and so
>>> on.
>>>>>>>>>>> 
>>>>>>>>>>> Ideas?
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> 
>>>>>>>>>>> Jean
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Nicolas Kourtellis
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 
>> 
>> 
>> --
>> Nicolas Kourtellis
>> 


Mime
View raw message