spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From diplomatic Guru <diplomaticg...@gmail.com>
Subject Re: [MLlib] How to set Loss to Gradient Boosted Tree in Java
Date Mon, 29 Feb 2016 16:41:18 GMT
Thank you very much Kevin.



On 29 February 2016 at 16:20, Kevin Mellott <kevin.r.mellott@gmail.com>
wrote:

> I found a helper class that I think should do the trick. Take a look at
> https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/tree/loss/Losses.scala
>
> When passing the Loss, you should be able to do something like:
>
> Losses.fromString("leastSquaresError")
>
> On Mon, Feb 29, 2016 at 10:03 AM, diplomatic Guru <
> diplomaticguru@gmail.com> wrote:
>
>> It's strange as you are correct the doc does state it. But it's
>> complaining about the constructor.
>>
>> When I clicked on the org.apache.spark.mllib.tree.loss.AbsoluteError
>> class, this is what I see:
>>
>>
>> @Since("1.2.0")
>> @DeveloperApi
>> object AbsoluteError extends Loss {
>>
>>   /**
>>    * Method to calculate the gradients for the gradient boosting
>> calculation for least
>>    * absolute error calculation.
>>    * The gradient with respect to F(x) is: sign(F(x) - y)
>>    * @param prediction Predicted label.
>>    * @param label True label.
>>    * @return Loss gradient
>>    */
>>   @Since("1.2.0")
>>   override def gradient(prediction: Double, label: Double): Double = {
>>     if (label - prediction < 0) 1.0 else -1.0
>>   }
>>
>>   override private[mllib] def computeError(prediction: Double, label:
>> Double): Double = {
>>     val err = label - prediction
>>     math.abs(err)
>>   }
>> }
>>
>>
>> On 29 February 2016 at 15:49, Kevin Mellott <kevin.r.mellott@gmail.com>
>> wrote:
>>
>>> Looks like it should be present in 1.3 at
>>> org.apache.spark.mllib.tree.loss.AbsoluteError
>>>
>>>
>>> spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/tree/loss/AbsoluteError.html
>>>
>>> On Mon, Feb 29, 2016 at 9:46 AM, diplomatic Guru <
>>> diplomaticguru@gmail.com> wrote:
>>>
>>>> AbsoluteError() constructor is undefined.
>>>>
>>>> I'm using Spark 1.3.0, maybe it is not ready for this version?
>>>>
>>>>
>>>>
>>>> On 29 February 2016 at 15:38, Kevin Mellott <kevin.r.mellott@gmail.com>
>>>> wrote:
>>>>
>>>>> I believe that you can instantiate an instance of the AbsoluteError
>>>>> class for the *Loss* object, since that object implements the Loss
>>>>> interface. For example.
>>>>>
>>>>> val loss = new AbsoluteError()
>>>>> boostingStrategy.setLoss(loss)
>>>>>
>>>>> On Mon, Feb 29, 2016 at 9:33 AM, diplomatic Guru <
>>>>> diplomaticguru@gmail.com> wrote:
>>>>>
>>>>>> Hi Kevin,
>>>>>>
>>>>>> Yes, I've set the bootingStrategy like that using the example. But
>>>>>> I'm not sure how to create and pass the Loss object.
>>>>>>
>>>>>> e.g
>>>>>>
>>>>>> boostingStrategy.setLoss(......);
>>>>>>
>>>>>> Not sure how to pass the selected Loss.
>>>>>>
>>>>>> How do I set the  Absolute Error in setLoss() function?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 29 February 2016 at 15:26, Kevin Mellott <
>>>>>> kevin.r.mellott@gmail.com> wrote:
>>>>>>
>>>>>>> You can use the constructor that accepts a BoostingStrategy object,
>>>>>>> which will allow you to set the tree strategy (and other hyperparameters
as
>>>>>>> well).
>>>>>>>
>>>>>>> *GradientBoostedTrees
>>>>>>> <http://spark.apache.org/docs/latest/api/java/org/apache/spark/mllib/tree/GradientBoostedTrees.html#GradientBoostedTrees(org.apache.spark.mllib.tree.configuration.BoostingStrategy)>*
>>>>>>> (BoostingStrategy
>>>>>>> <http://spark.apache.org/docs/latest/api/java/org/apache/spark/mllib/tree/configuration/BoostingStrategy.html>
>>>>>>>  boostingStrategy)
>>>>>>>
>>>>>>> On Mon, Feb 29, 2016 at 9:21 AM, diplomatic Guru <
>>>>>>> diplomaticguru@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello guys,
>>>>>>>>
>>>>>>>> I think the default Loss algorithm is Squared Error for regression,
>>>>>>>> but how do I change that to Absolute Error in Java.
>>>>>>>>
>>>>>>>> Could you please show me an example?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message