spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanbo Liang <yblia...@gmail.com>
Subject Re: Survival Curves using AFT implementation in Spark
Date Fri, 26 Feb 2016 10:40:03 GMT
Hi Stuti,

AFTSurvivalRegression does not support computing the predicted survival
functions/curves currently.
I don't know whether the quantile predictions can help you, you can refer
the example
<https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/ml/AFTSurvivalRegressionExample.scala#L48>
.
Maybe we can add this feature later.

Thanks
Yanbo

2016-02-26 14:35 GMT+08:00 Stuti Awasthi <stutiawasthi@hcl.com>:

> Hi All,
>
> I wanted to apply Survival Analysis using Spark AFT algorithm
> implementation. Now I perform the same in R using coxph model and passing
> the model in Survfit() function to generate survival curves
>
> Then I can visualize the survival curve on validation data to understand
> how good my model fits.
>
>
>
> R: Code
>
> fit <- coxph(Surv(futime, fustat) ~ age, data = ovarian)
>
> plot(survfit(fit,newdata=data.frame(age=60)))
>
>
>
> I wanted to achieve something similar with Spark. Hence I created the AFT
> model using Spark and passed my Test dataframe for prediction. The result
> of prediction is single prediction value for single input data which is as
> expected. But now how can I use this model to generate the Survival curves
> for visualization.
>
>
>
> Eg: Spark Code model.transform(test_final).show()
>
>
>
> standardized_features|       prediction|
>
> +---------------------+-----------------+
>
> | [0.0,0.0,0.743853...|48.33071792204102|
>
> +---------------------+-----------------+
>
>
>
> Can any suggest how to use the developed model for plotting Survival
> Curves for “test_final” data which is a dataframe feature[vector].
>
>
>
> Thanks
>
> Stuti Awasthi
>
>
>
>
>
> ::DISCLAIMER::
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>

Mime
View raw message