samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (SAMOA-68) Saving true and predicted labels to file
Date Wed, 21 Jun 2017 14:20:00 GMT


ASF GitHub Bot commented on SAMOA-68:

Github user nicolas-kourtellis commented on the issue:
    Thank you @mgrzenda for the contribution! Very nice!
    I tested it with bagging and VHT, covtypeNorm and random-tree-generator.
    Quick feedback:
    - Can you please remove the copyright years etc., as per the other PRs?
    - Can you add some comments/help on what each of the new parameters mean?
    - I think the new parameter on the frequency is not included in the defined templates.
Can you check and add it?
    - I think the printout of the last prediction is not printed in file. Not such a big deal
but please check if there is something you can do to fix it (or if it's something bigger).
    - Interestingly, the higher the frequency of predicted labels printed (controlled by -h),
the higher the overhead on the execution, and the longer it takes to finish the examples I
tried. However, practically when outputing every say 10 instances, the impact is minimal.
Any way we can make it impact less the execution time? Or is this due to I/O bottleneck to

> Saving true and predicted labels to file
> ----------------------------------------
>                 Key: SAMOA-68
>                 URL:
>             Project: SAMOA
>          Issue Type: New Feature
>          Components: SAMOA-API
>            Reporter: Maciej Grzenda
>              Labels: features
> Currently PrequentialEvaluation task supports dumpFile option.  With this option model
performance can be saved to a file. However, in some cases it would be good to save also individual
predictions made by a model.  This is useful for model debugging and method development.
> This could be also used to visualize model output, calculate custom performance indicators
(e.g. model accuracy for instances of a certain class or sharing the same feature value).
 Such saving of model output (if done) should be made for every instance. Hence, a new option
making it possible to dump predictions to a separate file seems justified.  For classification,
it should include votes made for individual classes, if available.

This message was sent by Atlassian JIRA

View raw message