ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuriy Babak <y.ch...@gmail.com>
Subject Re: [ML] Deployment of user-defined preprocessors
Date Wed, 05 Jun 2019 14:19:42 GMT
Alexey,

This is a cool change, do you create a ticket for it?

If no I can create one.

Best regards,
Yuriy Babak


пт, 31 мая 2019 г. в 14:20, Алексей Платонов <aplatonovv@gmail.com>:

> Hi, Igniters!
> Currently we don't have an ability to deploy automatically user-defined
> preprocessors and vectorizers. Client's code should be deployed manually to
> Ignite server nodes.
>
> I have an idea how to fix it. If we pass user's classloader and one of
> user-defined classes from fit-level to
> ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> GridPeerDeployAware interface to send informtation about this classloader
> to server nodes.
>
> To support this ability we can define interfaces like these:
>
> public interface DeployableObject {
>     public List<Object> getDependencies();
> }
>
> and
>
> public interface DeployingContext {
>     public Class<?> userClass();
>     public ClassLoader clientClassLoader();
> }
>
> DeployableObject will be mark for our ignite-ml final classes like trainers
> or concrete preprocessors and it can be able to return all dependencies
> that should be deployed to server nodes if it's needed. If these
> dependencies are DeployableObjects too then depenndencies will be unfolded
> recursively. Classes that isn't defined as DeployableObject will be
> recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
> DeployableObject).
>
> This list of DeployableObjects will be user for define user class loader
> and one of these objects will be used for passing to GridPeerDeployAware.
>
> So, this logic allows to pass user-defined Preprocessors and Vectorizers to
> training algorithms and pipelines.
>
> What do you think?
>
> Sincerely
> Alexey Platonov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message