spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debasish Das <debasish.da...@gmail.com>
Subject Hinge Gradient
Date Wed, 13 Dec 2017 08:20:46 GMT
Hi,

I looked into the LinearSVC flow and found the gradient for hinge as
follows:

Our loss function with {0, 1} labels is max(0, 1 - (2y - 1) (f_w(x)))
Therefore the gradient is -(2y - 1)*x

max is a non-smooth function.

Did we try using ReLu/Softmax function and use that to smooth the hinge
loss ?

Loss function will change to SoftMax(0, 1 - (2y-1) (f_w(x)))

Since this function is smooth, gradient will be well defined and
LBFGS/OWLQN should behave well.

Please let me know if this has been tried already. If not I can run some
benchmarks.

We have soft-max in multinomial regression and can be reused for LinearSVC
flow.

Thanks.
Deb

Mime
View raw message