spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tanyinyan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6349) Add probability estimates in SVMModel predict result
Date Mon, 16 Mar 2015 07:08:39 GMT
tanyinyan created SPARK-6349:
--------------------------------

             Summary: Add probability estimates in SVMModel predict result
                 Key: SPARK-6349
                 URL: https://issues.apache.org/jira/browse/SPARK-6349
             Project: Spark
          Issue Type: New Feature
          Components: MLlib
    Affects Versions: 1.2.1
            Reporter: tanyinyan


In SVMModel, predictPoint method output raw margin(threshold not set) or 1/0 label(threshold
set). 

when SVM are used as a classifier, it's hard to find a good threshold,and the raw margin is
hard to understand. 

when I am using SVM on dataset(https://www.kaggle.com/c/avazu-ctr-prediction/data), train
on the first day's dataset(ignore field id/device_id/device_ip, all remaining fields are concidered
as categorical variable, and sparsed before SVM) and predict on the same data with threshold
cleared, the predict result are all  negative. I have to set threshold to -1 to get a reasonable
confusion matrix.

So, I suggest to provide probability predict result in SVMModel as in libSVM(Platt's binary
SVM Probablistic Output)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message