samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduardo Costa <eduardoc...@gmail.com>
Subject Re: SAMOA stopping before the end of the dataset
Date Thu, 15 Sep 2016 03:06:19 GMT
Hi Nicolas!

One more test: now, only SAMOA, without STORM:

4)  Dataset with 107000 instances, 30 attributes and
"classifiers.ensemble.AdaptiveBagging -s 2"

command:

bin/samoa local target/SAMOA-Local-0.4.0-incubating-SNAPSHOT.jar
"PrequentialEvaluation -i -1 -f 1000 -d
/tmp/cenao11-2classes-ordenado-trat_entdv-sem-label-puro.csv
-l (classifiers.ensemble.AdaptiveBagging -s 1) -s
(org.apache.samoa.streams.ArffFileStream -s HDFSFileStreamSource -f
/cenario11-2classes-ordenado-trat_entdv-sem-label-puro.csv.arff)"

Output:

2016-09-13 23:16:30,840 [main] INFO
org.apache.samoa.evaluation.EvaluatorProcessor
(EvaluatorProcessor.java:83) - 59 seconds for 1000 instances
2016-09-13 23:16:30,841 [main] INFO
org.apache.samoa.evaluation.EvaluatorProcessor
(EvaluatorProcessor.java:169) - evaluation instances = 54,000
classified instances = 54,000
classifications correct (percent) = 99.998
Kappa Statistic (percent) = 0
Kappa Temporal Statistic (percent) = 50
Exception in thread "main" java.lang.NullPointerException
	at org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor.process(ModelAggregatorProcessor.java:141)
	at org.apache.samoa.topology.impl.SimpleProcessingItem.processEvent(SimpleProcessingItem.java:84)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:72)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:61)
	at org.apache.samoa.learners.classifiers.trees.FilterProcessor.process(FilterProcessor.java:87)
	at org.apache.samoa.topology.impl.SimpleProcessingItem.processEvent(SimpleProcessingItem.java:84)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:72)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:61)
	at org.apache.samoa.learners.classifiers.ensemble.BaggingDistributorProcessor.process(BaggingDistributorProcessor.java:82)
	at org.apache.samoa.topology.impl.SimpleProcessingItem.processEvent(SimpleProcessingItem.java:84)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:72)
	at org.apache.samoa.topology.impl.SimpleStream.put(SimpleStream.java:61)
	at org.apache.samoa.topology.LocalEntranceProcessingItem.injectNextEvent(LocalEntranceProcessingItem.java:45)
	at org.apache.samoa.topology.LocalEntranceProcessingItem.startSendingEvents(LocalEntranceProcessingItem.java:63)
	at org.apache.samoa.topology.impl.SimpleTopology.run(SimpleTopology.java:44)
	at org.apache.samoa.topology.impl.SimpleEngine.submitTopology(SimpleEngine.java:33)
	at org.apache.samoa.LocalDoTask.main(LocalDoTask.java:88)

Sorry, but I forgot to get the system status :-)


Thanks,

Eduardo.


2016-09-13 22:47 GMT-03:00 Eduardo Costa <eduardocosi@gmail.com>:

> Hi Nicolas,
>
> I did the fllowing tests so far:
>
> 1)  Dataset with 1309000 instances, 30 attributes and
> "classifiers.ensemble.Bagging -s 2"
>
> command:
>
> bin/samoa storm target/SAMOA-Storm-0.4.0-incubating-SNAPSHOT.jar
> "PrequentialEvaluation -i -1 -f 1000 -d /tmp/cenario10-2classes-ordena
> do-trat_entdv-sem-label-puro.csv -l (classifiers.ensemble.Bagging -s 2)
> -s (org.apache.samoa.streams.ArffFileStream -s HDFSFileStreamSource -f
> /cenario10-2classes-ordenado-trat_entdv-sem-label-puro.csv.arff)"
>
>
> status before classification:
>
> Tasks: 139 total,   1 running, 138 sleeping,   0 stopped,   0 zombie
> %Cpu(s):  0.5 us,  0.1 sy,  0.0 ni, 99.4 id,  0.0 wa,  0.0 hi,  0.0 si,
>  0.0 st
> KiB Mem:  30951404 total,  2327304 used, 28624100 free,    90060 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  1211872 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>  3947 root      20   0 2770m 236m  18m S   1.0  0.8   0:16.06 java
>
>  2852 root      20   0  199m  41m 6848 S   0.3  0.1   0:05.99 Xorg
>
>  3792 root      20   0 2569m 202m  18m S   0.3  0.7   0:04.81 java
>
>     1 root      20   0 10648  824  688 S   0.0  0.0   0:00.72 init
>
> status during classification (success):
>
> Tasks: 142 total,   1 running, 141 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 75.1 us, 23.4 sy,  0.0 ni,  1.3 id,  0.0 wa,  0.0 hi,  0.3 si,
>  0.0 st
> KiB Mem:  30951404 total, 10100004 used, 20851400 free,    90768 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  1873500 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>  4713 root      20   0 9467m 6.7g  15m S 391.1 22.8   4:14.95 java
>
>  3462 root      20   0  305m  18m  11m S   1.3  0.1   0:04.70
> gnome-terminal
>  3947 root      20   0 2770m 236m  18m S   1.3  0.8   0:18.42 java
>
>    19 root      20   0     0    0    0 S   0.3  0.0   0:00.11 ksoftirqd/3
>
> output:
>
> 1309000.0,1309000.0,93.50015278838808,0.0,-249.044141778799
>
> 196069 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1] INFO
>  org.apache.samoa.evaluation.EvaluatorProcessor - total evaluation time:
> 145 seconds for 1309791 instances
> 230680 [main] INFO  backtype.storm.daemon.nimbus - Delaying event :remove
> for 30 secs for Prequential_20160913213245-1-1473813184
>
> 2)  Dataset with 107000 instances, 30 attributes and
> "classifiers.ensemble.AdaptiveBagging -s 1"
>
> command:
>
> bin/samoa storm target/SAMOA-Storm-0.4.0-incubating-SNAPSHOT.jar
> "PrequentialEvaluation -i -1 -f 1000 -d /tmp/cenao11-2classes-ordenado-trat_entdv-sem-label-puro.csv
> -l (classifiers.ensemble.AdaptiveBagging -s 1) -s
> (org.apache.samoa.streams.ArffFileStream -s HDFSFileStreamSource -f
> /cenario11-2classes-ordenado-trat_entdv-sem-label-puro.csv.arff)"
>
> status before classification:
>
> Tasks: 140 total,   1 running, 139 sleeping,   0 stopped,   0 zombie
> %Cpu(s):  0.4 us,  0.1 sy,  0.0 ni, 99.5 id,  0.0 wa,  0.0 hi,  0.0 si,
>  0.0 st
> KiB Mem:  30951404 total,  3150588 used, 27800816 free,    92904 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  1981304 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>  3947 root      20   0 2770m 248m  18m S   2.3  0.8   0:26.20 java
>
>  3630 root      20   0 2616m 265m  18m S   0.3  0.9   0:11.40 java
>
>     1 root      20   0 10648  824  688 S   0.0  0.0   0:00.73 init
>
>     2 root      20   0     0    0    0 S   0.0  0.0   0:00.00 kthreadd
>
>
> status during classification (fail):
>
> Tasks: 145 total,   1 running, 144 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 74.1 us, 24.2 sy,  0.0 ni,  1.1 id,  0.0 wa,  0.0 hi,  0.7 si,
>  0.0 st
> KiB Mem:  30951404 total,  5737076 used, 25214328 free,   103376 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  3154772 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>
>  6601 root      20   0 9473m 1.2g  15m S 385.9  4.1   0:37.38 java
>
>
>  2852 root      20   0  200m  41m 6892 S   3.7  0.1   0:26.38 Xorg
>
>
>  3462 root      20   0  306m  18m  11m S   3.0  0.1   0:15.39
> gnome-terminal
>
>  3947 root      20   0 2775m 254m  18m S   1.0  0.8   0:53.04 java
>
>
>    19 root      20   0     0    0    0 S   0.3  0.0   0:00.72 ksoftirqd/3
>
>
>
> output:
>
> 15421 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1] INFO
>  org.apache.samoa.evaluation.EvaluatorProcessor - 0 seconds for 1000
> instances
> 15422 [Thread-22-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_1]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 15422 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1] INFO
>  org.apache.samoa.evaluation.EvaluatorProcessor - evaluation instances =
> 92,000
> classified instances = 92,000
> classifications correct (percent) = 98.609
> Kappa Statistic (percent) = 16.772
> Kappa Temporal Statistic (percent) = 22.892
> 15422 [Thread-22-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_1]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 15426 [Thread-22-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_1]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
>
>
>
>
> 3)  Dataset with 107000 instances, 30 attributes and
> "classifiers.ensemble.AdaptiveBagging -s 2"
>
> command:
>
> bin/samoa storm target/SAMOA-Storm-0.4.0-incubating-SNAPSHOT.jar
> "PrequentialEvaluation -i -1 -f 1000 -d /tmp/cenao11-2classes-ordenado-trat_entdv-sem-label-puro.csv
> -l (classifiers.ensemble.AdaptiveBagging -s 2) -s
> (org.apache.samoa.streams.ArffFileStream -s HDFSFileStreamSource -f
> /cenario11-2classes-ordenado-trat_entdv-sem-label-puro.csv.arff)"
>
> status before classification:
>
> Tasks: 140 total,   1 running, 139 sleeping,   0 stopped,   0 zombie
> %Cpu(s):  0.4 us,  0.1 sy,  0.0 ni, 99.5 id,  0.0 wa,  0.0 hi,  0.0 si,
>  0.0 st
> KiB Mem:  30951404 total,  3150588 used, 27800816 free,    92904 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  1981304 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>  3947 root      20   0 2770m 248m  18m S   2.3  0.8   0:26.20 java
>
>  3630 root      20   0 2616m 265m  18m S   0.3  0.9   0:11.40 java
>
>     1 root      20   0 10648  824  688 S   0.0  0.0   0:00.73 init
>
>     2 root      20   0     0    0    0 S   0.0  0.0   0:00.00 kthreadd
>
>
>
>
> status during classification (fail):
>
> Tasks: 142 total,   1 running, 141 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 78.1 us, 20.4 sy,  0.0 ni,  1.2 id,  0.0 wa,  0.0 hi,  0.3 si,
>  0.0 st
> KiB Mem:  30951404 total,  5056400 used, 25895004 free,    94752 buffers
> KiB Swap:   901116 total,        0 used,   901116 free,  2189600 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>
>
>  5641 root      20   0 9494m 1.6g  15m S 391.3  5.5   0:41.70 java
>
>
>  3462 root      20   0  305m  18m  11m S   2.0  0.1   0:09.75
> gnome-terminal
>
>  3947 root      20   0 2770m 248m  18m S   1.0  0.8   0:30.81 java
>
>
>    11 root      20   0     0    0    0 S   0.3  0.0   0:00.40 kworker/0:1
>
>
>
> output:
>
> 18985 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1] INFO
>  org.apache.samoa.evaluation.EvaluatorProcessor - 0 seconds for 1000
> instances
> 18985 [Thread-28-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_2]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 18985 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1] INFO
>  org.apache.samoa.evaluation.EvaluatorProcessor - evaluation instances =
> 85,000
> classified instances = 85,000
> classifications correct (percent) = 99.531
> Kappa Statistic (percent) = 8.933
> Kappa Temporal Statistic (percent) = 21.457
> 18987 [Thread-28-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_2]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 18987 [Thread-28-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_2]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 18988 [Thread-28-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_2]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
> 18989 [Thread-28-org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor_2]
> INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor
> - Change detected, resetting the classifier
>
>
> Note: I used the same number of attributes to facilitate the comparison.
> System configuration:
> Debian 7 (virtual machine)
> 30 GB RAM
> 4 processors
>
>
> Thanks,
> Eduardo.
>
>
>
>
>
> 2016-09-12 4:51 GMT-03:00 Nicolas Kourtellis <nkourtellis@gmail.com>:
>
>> Hi Eduardo,
>>
>> Thanks for the tests and questions,
>>
>> I can't tell if this is a problem of Storm / Nimbus or Samoa.
>> Have you tried a smaller stream, i.e., with a smaller number of events, or
>> smaller number of attributes?
>>
>> Perhaps the DSPE is shutting down due to memory error or other resource
>> issue?
>>
>> Try with different sizes and let us know the status of the machine before
>> it fails.
>>
>> Thanks,
>>
>> Nicolas
>>
>>
>>
>>
>> On Mon, Sep 12, 2016 at 4:25 AM, Eduardo Costa <eduardocosi@gmail.com>
>> wrote:
>>
>> > Dear,
>> > I'm executanto the Samoa with the storm in local mode (command line:
>> > bin/samoa storm target/SAMOA-Storm-0.4.0-incubating-SNAPSHOT.jar
>> > "PrequentialEvaluation -i -1 -f 1000 -d /tmp/cenario10-2classes-
>> > ordenado-trat_entdv-sem-label-puro.csv -l (classifiers.ensemble.
>> > AdaptiveBagging
>> > -s 1) -s (org.apache.samoa.streams.ArffFileStream -s
>> HDFSFileStreamSource
>> > -f /cenario10-2classes-ordenado-trat_entdv-sem-label-puro.csv.arff)").
>> >
>> > I am using a dataset of 1,300.00 instances, but this samoa stopping at
>> > 480,000 when I use Adaptive Bagging. After a while the following message
>> > appears:
>> >
>> > ------------------------------------------------------------
>> > ---------------------------------------------------
>> >
>> > org.apache.samoa.learners.classifiers.trees.ModelAggregatorProcessor -
>> > Change detected, resetting the classifier
>> > 66025 [Thread-22-org.apache.samoa.learners.classifiers.trees.
>> > ModelAggregatorProcessor_1]
>> > INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorP
>> rocessor
>> > - Change detected, resetting the classifier
>> > 66025 [Thread-22-org.apache.samoa.learners.classifiers.trees.
>> > ModelAggregatorProcessor_1]
>> > INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorP
>> rocessor
>> > - Change detected, resetting the classifier
>> > 66026 [Thread-22-org.apache.samoa.learners.classifiers.trees.
>> > ModelAggregatorProcessor_1]
>> > INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorP
>> rocessor
>> > - Change detected, resetting the classifier
>> > 66027 [Thread-22-org.apache.samoa.learners.classifiers.trees.
>> > ModelAggregatorProcessor_1]
>> > INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorP
>> rocessor
>> > - Change detected, resetting the classifier
>> > 66027 [Thread-22-org.apache.samoa.learners.classifiers.trees.
>> > ModelAggregatorProcessor_1]
>> > INFO  org.apache.samoa.learners.classifiers.trees.ModelAggregatorP
>> rocessor
>> > - Change detected, resetting the classifier
>> > 224126 [main] INFO  backtype.storm.daemon.nimbus - Delaying event
>> :remove
>> > for 30 secs for Prequential_20160911213510-1-1473640527
>> > 224132 [main] INFO  backtype.storm.daemon.nimbus - Updated
>> > Prequential_20160911213510-1-1473640527 with status {:type :killed,
>> > :kill-time-secs 30}
>> > 224132 [main] INFO  backtype.storm.daemon.nimbus - Shutting down master
>> > 224136 [ProcessThread(sid:0 cport:-1):] INFO
>> > org.apache.storm.zookeeper.server.PrepRequestProcessor
>> > - Processed session termination for sessionid: 0x1571bd1c9c50001
>> > 224137 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO
>> >  org.apache.storm.zookeeper.server.NIOServerCnxn - Closed socket
>> > connection
>> > for client /0:0:0:0:0:0:0:1:50353 which had sessionid 0x1571bd1c9c50001
>> > 224137 [main] INFO  org.apache.storm.zookeeper.ZooKeeper - Session:
>> > 0x1571bd1c9c50001 closed
>> > 224137 [main-EventThread] INFO  org.apache.storm.zookeeper.ClientCnxn -
>> > EventThread shut down
>> > 224142 [main] INFO  backtype.storm.daemon.nimbus - Shut down master
>> > 224155 [ProcessThread(sid:0 cport:-1):] INFO
>> > org.apache.storm.zookeeper.server.PrepRequestProcessor
>> > - Processed session termination for sessionid: 0x1571bd1c9c50003
>> > 224159 [main] INFO  org.apache.storm.zookeeper.ZooKeeper - Session:
>> > 0x1571bd1c9c50003 closed
>> > 224159 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO
>> >  org.apache.storm.zookeeper.server.NIOServerCnxn - Closed socket
>> > connection
>> > for client /0:0:0:0:0:0:0:1:50355 which had sessionid 0x1571bd1c9c50003
>> > 224159 [main-EventThread] INFO  org.apache.storm.zookeeper.ClientCnxn -
>> > EventThread shut down
>> > 224163 [ProcessThread(sid:0 cport:-1):] INFO
>> > org.apache.storm.zookeeper.server.PrepRequestProcessor
>> > - Processed session termination for sessionid: 0x1571bd1c9c50005
>> > 224164 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO
>> >  org.apache.storm.zookeeper.server.NIOServerCnxn - Closed socket
>> > connection
>> > for client /127.0.0.1:49569 which had sessionid 0x1571bd1c9c50005
>> > 224165 [main-EventThread] INFO  org.apache.storm.zookeeper.ClientCnxn -
>> > EventThread shut down
>> > 224165 [main] INFO  org.apache.storm.zookeeper.ZooKeeper - Session:
>> > 0x1571bd1c9c50005 closed
>> > 224166 [main] INFO  backtype.storm.daemon.supervisor - Shutting down
>> > 9957d6b3-05c3-46d4-bb6f-c37751a40026:37ed88ae-36f0-4b87-
>> b128-cd16c1fa21f7
>> > 224166 [main] INFO  backtype.storm.process-simulator - Killing process
>> > 3a39d80e-10b5-4db7-823d-a48ca5d85109
>> > 224166 [main] INFO  backtype.storm.daemon.worker - Shutting down worker
>> > Prequential_20160911213510-1-1473640527 9957d6b3-05c3-46d4-bb6f-
>> > c37751a40026
>> > 1024
>> > 224167 [main] INFO  backtype.storm.daemon.worker - Shutting down receive
>> > thread
>> > 224167 [main] INFO  backtype.storm.messaging.loader - Shutting down
>> > receiving-thread: [Prequential_20160911213510-1-1473640527, 1024]
>> > 224167 [main] INFO  backtype.storm.messaging.loader - Waiting for
>> > receiving-thread:[Prequential_20160911213510-1-1473640527, 1024] to die
>> > 224167 [Thread-10-worker-receiver-thread-0] INFO
>> > backtype.storm.messaging.loader
>> > - Receiving-thread:[Prequential_20160911213510-1-1473640527, 1024]
>> > received
>> > shutdown notice
>> > 224168 [main] INFO  backtype.storm.messaging.loader - Shutdown
>> > receiving-thread: [Prequential_20160911213510-1-1473640527, 1024]
>> > 224168 [main] INFO  backtype.storm.daemon.worker - Shut down receive
>> thread
>> > 224169 [main] INFO  backtype.storm.daemon.worker - Terminating messaging
>> > context
>> > 224169 [main] INFO  backtype.storm.daemon.worker - Shutting down
>> executors
>> > 224169 [main] INFO  backtype.storm.daemon.executor - Shutting down
>> executor
>> > org.apache.samoa.evaluation.EvaluatorProcessor_1:[2 2]
>> > 224169 [Thread-12-org.apache.samoa.evaluation.EvaluatorProcessor_1]
>> INFO
>> >  backtype.storm.util - Async loop interrupted!
>> > 224170 [Thread-11-disruptor-executor[2 2]-send-queue] INFO
>> >  backtype.storm.util - Async loop interrupted!
>> > 224170 [main] INFO  backtype.storm.daemon.executor - Shut down executor
>> > org.apache.samoa.evaluation.EvaluatorProcessor_1:[2 2]
>> > 224170 [main] INFO  backtype.storm.daemon.executor - Shutting down
>> executor
>> > org.apache.samoa.learners.classifiers.ensemble.
>> > BaggingDistributorProcessor_1:[3 3]
>> > ------------------------------------------------------------
>> > --------------------------------------------------------
>> > What can it be?
>> >
>> > Regards,
>> > Eduardo Costa.
>> >
>>
>>
>>
>> --
>> Nicolas Kourtellis
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message