ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: Thin client: compute support
Date Fri, 22 Nov 2019 15:40:39 GMT
> Two-way requests (from clients to server and from server to clients)
Alex, can you please clarify the need for server->client requests?

> compute tasks in any language that implemented thin client protocol
(python as a first candidate)
Nikolay, do you have anything in mind here? Why Python?




On Fri, Nov 22, 2019 at 3:13 PM Sergey Kozlov <skozlov@gridgain.com> wrote:

> Hi
>
> On Thu, Nov 21, 2019 at 12:48 PM Denis Garus <garus.d.g@gmail.com> wrote:
>
> > Hello, Sergey!
> >
> >
> > >> 3. Ignite doesn't have roles/authorization functionality for now.
> >
> >
> > I can't agree with you.
> >
>
> My bad, you're right. But the IEP page provided by you is limited and
> there's no clear how it works for thin clients
>
>
> > We already have authorization functionality in Ignite and for a thin
> > client too [1].
> >
> > But, compute support for a thin client requires some additional efforts
> to
> > get an appropriate SecurityContext on a remote node.
> >
> > The list of tasks allowed for subjects, including thin clients, is the
> > area of responsibility of GridSecurityProcessor [2].
> >
> >
> >
> >    1.
> >
> org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest
> >    2.
> org.apache.ignite.internal.processors.security.GridSecurityProcessor
> >
> >
> > чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn <ptupitsyn@apache.org>:
> >
> >> Good points, Sergey.
> >> Maybe you are right, and Java-based compute without peer deployment is a
> >> good first step for thin clients.
> >>
> >> On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov <skozlov@gridgain.com>
> >> wrote:
> >>
> >> > Hi Pavel
> >> >
> >> > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn <ptupitsyn@apache.org
> >
> >> > wrote:
> >> >
> >> > > 1. I believe that Cluster operations for Thin Client protocol are
> >> already
> >> > > in the works
> >> > > by Alexandr Shapkin. Can't find the ticket though.
> >> > > Alexandr, can you please confirm and attach the ticket number?
> >> > >
> >> > > 2. Proposed changes will work only for Java tasks that are already
> >> > deployed
> >> > > on server nodes.
> >> > > This is mostly useless for other thin clients we have (Python, PHP,
> >> .NET,
> >> > > C++).
> >> > >
> >> >
> >> > I don't guess so. The task (execution) is a way to implement own layer
> >> for
> >> > the thin client application.
> >> >
> >> >
> >> > > We should think of a way to make this useful for all clients.
> >> > > For example, we may allow sending tasks in some scripting language
> >> like
> >> > > Javascript.
> >> > > Thoughts?
> >> > >
> >> >
> >> > The arbitrary code execution from a remote client must be protected
> >> > from malicious code.
> >> > I don't know how it could be designed but without that we open the
> hole
> >> to
> >> > kill cluster.
> >> >
> >> >
> >> > >
> >> > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov <
> skozlov@gridgain.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Alex
> >> > > >
> >> > > > The idea is great. But I have some concerns that probably should
> be
> >> > taken
> >> > > > into account for design:
> >> > > >
> >> > > >    1. We need to have the ability to stop a task execution, smth
> >> like
> >> > > >    OP_COMPUTE_CANCEL_TASK  operation (client to server)
> >> > > >    2. What's about task execution timeout? It may help to the
> >> cluster
> >> > > >    survival for buggy tasks
> >> > > >    3. Ignite doesn't have roles/authorization functionality for
> now.
> >> > But
> >> > > a
> >> > > >    task is the risky operation for cluster (for security reasons).
> >> > Could
> >> > > we
> >> > > >    add for Ignite configuration new options:
> >> > > >       - Explicit turning on for compute task support for thin
> >> protocol
> >> > > >       (disabled by default) for whole cluster
> >> > > >       - Explicit turning on for compute task support for a node
> >> > > >       - The list of task names (classes) allowed to execute by
> thin
> >> > > client.
> >> > > >    4. Support the labeling for task that may help to investigate
> >> issues
> >> > > on
> >> > > >    cluster (the idea from IEP-34 [1])
> >> > > >
> >> > > > 1.
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov <
> >> > plehanov.alex@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hello, Igniters!
> >> > > > >
> >> > > > > I have plans to start implementation of Compute interface
for
> >> Ignite
> >> > > thin
> >> > > > > client and want to discuss features that should be implemented.
> >> > > > >
> >> > > > > We already have Compute implementation for binary-rest clients
> >> > > > > (GridClientCompute), which have the following functionality:
> >> > > > > - Filtering cluster nodes (projection) for compute
> >> > > > > - Executing task by the name
> >> > > > >
> >> > > > > I think we can implement this functionality in a thin client
as
> >> well.
> >> > > > >
> >> > > > > First of all, we need some operation types to request a
list of
> >> all
> >> > > > > available nodes and probably node attributes (by a list
of
> nodes).
> >> > Node
> >> > > > > attributes will be helpful if we will decide to implement
analog
> >> of
> >> > > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate
methods
> in
> >> > the
> >> > > > thin
> >> > > > > client. Perhaps they can be requested lazily.
> >> > > > >
> >> > > > > From the protocol point of view there will be two new
> operations:
> >> > > > >
> >> > > > > OP_CLUSTER_GET_NODES
> >> > > > > Request: empty
> >> > > > > Response: long topologyVersion, int minorTopologyVersion,
int
> >> > > nodesCount,
> >> > > > > for each node set of node fields (UUID nodeId, Object or
String
> >> > > > > consistentId, long order, etc)
> >> > > > >
> >> > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES
> >> > > > > Request: int nodesCount, for each node: UUID nodeId
> >> > > > > Response: int nodesCount, for each node: int attributesCount,
> for
> >> > each
> >> > > > node
> >> > > > > attribute: String name, Object value
> >> > > > >
> >> > > > > To execute tasks we need something like these methods in
the
> >> client
> >> > > API:
> >> > > > > Object execute(String task, Object arg)
> >> > > > > Future<Object> executeAsync(String task, Object arg)
> >> > > > > Object affinityExecute(String task, String cache, Object
key,
> >> Object
> >> > > arg)
> >> > > > > Future<Object> affinityExecuteAsync(String task, String
cache,
> >> Object
> >> > > > key,
> >> > > > > Object arg)
> >> > > > >
> >> > > > > Which can be mapped to protocol operations:
> >> > > > >
> >> > > > > OP_COMPUTE_EXECUTE_TASK
> >> > > > > Request: UUID nodeId, String taskName, Object arg
> >> > > > > Response: Object result
> >> > > > >
> >> > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY
> >> > > > > Request: String cacheName, Object key, String taskName,
Object
> arg
> >> > > > > Response: Object result
> >> > > > >
> >> > > > > The second operation is needed because we sometimes can't
> >> calculate
> >> > and
> >> > > > > connect to affinity node on the client-side (affinity awareness
> >> can
> >> > be
> >> > > > > disabled, custom affinity function can be used or there
can be
> no
> >> > > > > connection between client and affinity node), but we can
make
> best
> >> > > effort
> >> > > > > to send request to target node if affinity awareness is
enabled.
> >> > > > >
> >> > > > > Currently, on the server-side requests always processed
> >> synchronously
> >> > > and
> >> > > > > responses are sent right after request was processed. To
execute
> >> long
> >> > > > tasks
> >> > > > > async we should whether change this logic or introduce some
kind
> >> > > two-way
> >> > > > > communication between client and server (now only one-way
> requests
> >> > from
> >> > > > > client to server are allowed).
> >> > > > >
> >> > > > > Two-way communication can also be useful in the future if
we
> will
> >> > send
> >> > > > some
> >> > > > > server-side generated events to clients.
> >> > > > >
> >> > > > > In case of two-way communication there can be new operations
> >> > > introduced:
> >> > > > >
> >> > > > > OP_COMPUTE_EXECUTE_TASK (from client to server)
> >> > > > > Request: UUID nodeId, String taskName, Object arg
> >> > > > > Response: long taskId
> >> > > > >
> >> > > > > OP_COMPUTE_TASK_FINISHED (from server to client)
> >> > > > > Request: taskId, Object result
> >> > > > > Response: empty
> >> > > > >
> >> > > > > The same for affinity requests.
> >> > > > >
> >> > > > > Also, we can implement not only execute task operation,
but some
> >> > other
> >> > > > > operations from IgniteCompute (broadcast, run, call), but
it
> will
> >> be
> >> > > > useful
> >> > > > > only for java thin client. And even with java thin client
we
> >> should
> >> > > > whether
> >> > > > > implement peer-class-loading for thin clients (this also
> requires
> >> > > two-way
> >> > > > > client-server communication) or put classes with executed
> >> closures to
> >> > > the
> >> > > > > server locally.
> >> > > > >
> >> > > > > What do you think about proposed protocol changes?
> >> > > > > Do we need two-way requests between client and server?
> >> > > > > Do we need support of compute methods other than "execute
task"?
> >> > > > > What do you think about peer-class-loading for thin clients?
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Sergey Kozlov
> >> > > > GridGain Systems
> >> > > > www.gridgain.com
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > Sergey Kozlov
> >> > GridGain Systems
> >> > www.gridgain.com
> >> >
> >>
> >
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message