spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artemis User <arte...@dtechspace.com>
Subject Persisting Customized Transformer
Date Tue, 19 Jan 2021 19:29:55 GMT
We are trying to create a customized transformer for a ML pipeline and 
also want to persist the trained pipeline and retrieve it for 
production.  To enable persistency, we will have to implement read/write 
functions.  However, this is not feasible in Scala since the read/write 
methods are private members of the MLModel class.  This problem was 
described in a JIRA ticket https://issues.apache.org/jira/browse/SPARK-17048

Although the ticket suggested some workaround, but only in Java. I was 
wondering if anyone has tried in Scala?  The scala work-around suggested 
in the ticket didn't work for us.  Does anyone know if this issue has 
been resolved in 3.0.1 or the upcoming 3.1?  Any suggestion is highly 
appreciated.

-- ND


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message