ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: Deprecate IgniteRDD in embedded mode
Date Fri, 01 Dec 2017 20:41:22 GMT
Ticket created: https://issues.apache.org/jira/browse/IGNITE-7092

-Val

On Thu, Nov 30, 2017 at 5:01 PM, Denis Magda <dmagda@apache.org> wrote:

> Val,
>
> Sounds reasonable to me. The fewer useless and potentially harmful
> features or “switches” we have in Ignite the clear it will be for the user
> how to use us in a right way.
>
> +1 for the deprecation and further removal.
>
> —
> Denis
>
> > On Nov 30, 2017, at 3:07 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
> >
> > Igniters,
> >
> > Currently we claim to support IgniteRDD in two modes: standalone and
> > embedded. Standalone means there is a separately running Ignite cluster,
> > and Spark start client node(s) to interact with it. In embedded node
> > everything runs within Spark, including Ignite server nodes that are
> > started embedded into Spark executors.
> >
> > The latter case doesn't really work, mainly because the lifecycle of
> Spark
> > executors is not very predictable - Spark can start and stop them while
> > application is running. In case Ignite cluster is used to store data
> (which
> > is usually the case), this causes unnecessary rebalancing or even
> > unexpected data loss.
> >
> > I propose to deprecate and eventually discontinue the embedded mode.
> > Luckily, standalone mode is the default one, so we can simply print out a
> > clear warning if one switches to embedded mode, and also mention this in
> > the docs.
> >
> > Thoughts? If there are no objections, I will create a ticket and make the
> > change.
> >
> > -Val
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message