samoa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SAMOA-68) Saving true and predicted labels to file
Date Fri, 23 Jun 2017 14:29:01 GMT

    [ https://issues.apache.org/jira/browse/SAMOA-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060986#comment-16060986
] 

ASF GitHub Bot commented on SAMOA-68:
-------------------------------------

Github user abifet commented on a diff in the pull request:

    https://github.com/apache/incubator-samoa/pull/61#discussion_r123756844
  
    --- Diff: samoa-api/src/main/java/org/apache/samoa/evaluation/EvaluatorProcessor.java
---
    @@ -179,6 +211,28 @@ private void addMeasurement() {
         }
       }
     
    +  /**
    +   * This method is used to create one line of a text file containing predictions and
votes (for classification only).
    +   * In case, this is the first line a header line is also added
    +   */
    +  private void addVote() {
    +    List<Vote> votes = new Vector<>();
    +    Collections.addAll(votes, evaluator.getPredictionVotes());
    +    Vote[] finalVotes = votes.toArray(new Vote[votes.size()]);
    +    learningCurve.setVote(finalVotes);
    +    logger.debug("evaluator id = {}", this.id);
    +
    --- End diff --
    
    Do we need to store in LearningCurve the information of the votes? Can we print them directly
using finalVotes? learningCurve.voteEntryToString is only used here to output the votes, right?


> Saving true and predicted labels to file
> ----------------------------------------
>
>                 Key: SAMOA-68
>                 URL: https://issues.apache.org/jira/browse/SAMOA-68
>             Project: SAMOA
>          Issue Type: New Feature
>          Components: SAMOA-API
>            Reporter: Maciej Grzenda
>              Labels: features
>
> Currently PrequentialEvaluation task supports dumpFile option.  With this option model
performance can be saved to a file. However, in some cases it would be good to save also individual
predictions made by a model.  This is useful for model debugging and method development.
> This could be also used to visualize model output, calculate custom performance indicators
(e.g. model accuracy for instances of a certain class or sharing the same feature value).
 Such saving of model output (if done) should be made for every instance. Hence, a new option
making it possible to dump predictions to a separate file seems justified.  For classification,
it should include votes made for individual classes, if available.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message