spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From svattig <>
Subject GLM Poisson Model - Deviance calculations
Date Wed, 18 Apr 2018 19:37:32 GMT
In Spark 2.3, When Poisson Model(with labelCol having few counts as 0's) is
fit, the Deviance calculations are broken as result of log(0). I think this
is the same case as in spark 2.2. 
But the new toString method in Spark 2.3's
GeneralizedLinearRegressionTrainingSummary class is throwing error at line
1551 with NumberFormatException. Due to this exception, we are not able to
get the summary object from Model fit.

Can the toString method be fixed including Deviance calculations for example
taking log(1) when ever the count is 0 instead of having log(0) ?


Sent from:

To unsubscribe e-mail:

View raw message