spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yuhao yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-5384) Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different lengths
Date Fri, 23 Jan 2015 16:30:35 GMT
yuhao yang created SPARK-5384:
---------------------------------

             Summary: Vectors.sqdist return inconsistent result for sparse/dense vectors when
the vectors have different lengths
                 Key: SPARK-5384
                 URL: https://issues.apache.org/jira/browse/SPARK-5384
             Project: Spark
          Issue Type: Bug
          Components: MLlib
    Affects Versions: 1.2.1
         Environment: centos, others should be similar
            Reporter: yuhao yang
            Priority: Critical
             Fix For: 1.2.1


For two vectors of different lengths, Vectors.sqdist would return different result when the
vectors are represented as sparse and dense respectively. Sample:   
    val s1 = new SparseVector(4, Array(0,1,2,3), Array(1.0, 2.0, 3.0, 4.0))
    val s2 = new SparseVector(1, Array(0), Array(9.0))
    val d1 = new DenseVector(Array(1.0, 2.0, 3.0, 4.0))
    val d2 = new DenseVector(Array(9.0))
    println(s1 == d1 && s2 == d2)
    println(Vectors.sqdist(s1, s2))
    println(Vectors.sqdist(d1, d2))
result:
     true
     93.0
     64.0

More precisely, for the extra part, Vectors.sqdist would include it for sparse vectors and
exclude it for dense vectors. I'll send a PR and we can have more detailed discussion there.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message