ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Thin client: compute support
Date Fri, 22 Nov 2019 10:41:37 GMT
Hello!

When doing this, I think we should also consider Service invocations to
make their API/implementation as close as possible.

Regards,
-- 
Ilya Kasnacheev


пт, 22 нояб. 2019 г. в 11:17, Alex Plehanov <plehanov.alex@gmail.com>:

> So, guys:
>
> Cluster API (get nodes and get node attributes) will be implemented in
> scope of IGNITE-12385 ticket by Aleksandr.
>
> For compute API as a first step only the ability to execute the task by
> task name operation will be implemented, which will require:
> - Two-way requests (from clients to server and from server to clients)
> - Four new operation types: OP_COMPUTE_EXECUTE_TASK (client-server),
> OP_COMPUTE_EXECUTE_TASK_AFFINITY (client-server), OP_COMPUTE_CANCEL_TASK
> (client-server), OP_COMPUTE_TASK_FINISHED (server-client)
> - Option in node configuration to enable compute for thin clients (disabled
> by default)
>
> I think we should create IEP to describe protocol changes and new
> operations format. I will create it.
>
> Aleksandr, can you provide some details about your implementation? What
> types of new operations will be introduced? Can you describe the format of
> new operations? I will add this information to IEP.
>
> As a next step we can implement class deployment from thin clients (will
> work only for java) or some kind of domain specific language. It will be
> decided later do we really need this.
>
> WDYT?
>
> пт, 22 нояб. 2019 г. в 10:19, Alex Plehanov <plehanov.alex@gmail.com>:
>
> > Denis, the primary motivation is to enable execution of deployed to
> server
> > java tasks from thin clients (java and other languages).
> >
> > пт, 22 нояб. 2019 г. в 00:03, Denis Magda <dmagda@apache.org>:
> >
> >> Alex, what is the primary motivation for this improvement? Are you
> looking
> >> to enable the compute APIs for languages different from Java?
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Nov 20, 2019 at 11:58 PM Alex Plehanov <plehanov.alex@gmail.com
> >
> >> wrote:
> >>
> >> > Hello, Igniters!
> >> >
> >> > I have plans to start implementation of Compute interface for Ignite
> >> thin
> >> > client and want to discuss features that should be implemented.
> >> >
> >> > We already have Compute implementation for binary-rest clients
> >> > (GridClientCompute), which have the following functionality:
> >> > - Filtering cluster nodes (projection) for compute
> >> > - Executing task by the name
> >> >
> >> > I think we can implement this functionality in a thin client as well.
> >> >
> >> > First of all, we need some operation types to request a list of all
> >> > available nodes and probably node attributes (by a list of nodes).
> Node
> >> > attributes will be helpful if we will decide to implement analog of
> >> > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the
> >> thin
> >> > client. Perhaps they can be requested lazily.
> >> >
> >> > From the protocol point of view there will be two new operations:
> >> >
> >> > OP_CLUSTER_GET_NODES
> >> > Request: empty
> >> > Response: long topologyVersion, int minorTopologyVersion, int
> >> nodesCount,
> >> > for each node set of node fields (UUID nodeId, Object or String
> >> > consistentId, long order, etc)
> >> >
> >> > OP_CLUSTER_GET_NODE_ATTRIBUTES
> >> > Request: int nodesCount, for each node: UUID nodeId
> >> > Response: int nodesCount, for each node: int attributesCount, for each
> >> node
> >> > attribute: String name, Object value
> >> >
> >> > To execute tasks we need something like these methods in the client
> API:
> >> > Object execute(String task, Object arg)
> >> > Future<Object> executeAsync(String task, Object arg)
> >> > Object affinityExecute(String task, String cache, Object key, Object
> >> arg)
> >> > Future<Object> affinityExecuteAsync(String task, String cache, Object
> >> key,
> >> > Object arg)
> >> >
> >> > Which can be mapped to protocol operations:
> >> >
> >> > OP_COMPUTE_EXECUTE_TASK
> >> > Request: UUID nodeId, String taskName, Object arg
> >> > Response: Object result
> >> >
> >> > OP_COMPUTE_EXECUTE_TASK_AFFINITY
> >> > Request: String cacheName, Object key, String taskName, Object arg
> >> > Response: Object result
> >> >
> >> > The second operation is needed because we sometimes can't calculate
> and
> >> > connect to affinity node on the client-side (affinity awareness can be
> >> > disabled, custom affinity function can be used or there can be no
> >> > connection between client and affinity node), but we can make best
> >> effort
> >> > to send request to target node if affinity awareness is enabled.
> >> >
> >> > Currently, on the server-side requests always processed synchronously
> >> and
> >> > responses are sent right after request was processed. To execute long
> >> tasks
> >> > async we should whether change this logic or introduce some kind
> two-way
> >> > communication between client and server (now only one-way requests
> from
> >> > client to server are allowed).
> >> >
> >> > Two-way communication can also be useful in the future if we will send
> >> some
> >> > server-side generated events to clients.
> >> >
> >> > In case of two-way communication there can be new operations
> introduced:
> >> >
> >> > OP_COMPUTE_EXECUTE_TASK (from client to server)
> >> > Request: UUID nodeId, String taskName, Object arg
> >> > Response: long taskId
> >> >
> >> > OP_COMPUTE_TASK_FINISHED (from server to client)
> >> > Request: taskId, Object result
> >> > Response: empty
> >> >
> >> > The same for affinity requests.
> >> >
> >> > Also, we can implement not only execute task operation, but some other
> >> > operations from IgniteCompute (broadcast, run, call), but it will be
> >> useful
> >> > only for java thin client. And even with java thin client we should
> >> whether
> >> > implement peer-class-loading for thin clients (this also requires
> >> two-way
> >> > client-server communication) or put classes with executed closures to
> >> the
> >> > server locally.
> >> >
> >> > What do you think about proposed protocol changes?
> >> > Do we need two-way requests between client and server?
> >> > Do we need support of compute methods other than "execute task"?
> >> > What do you think about peer-class-loading for thin clients?
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message