spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamas Jambor <jambo...@gmail.com>
Subject Re: multiple sparkcontexts and streamingcontexts
Date Mon, 02 Mar 2015 15:01:47 GMT
thanks for the reply.

Actually, our main problem is not really about sparkcontext, the problem is
that spark does not allow to create streaming context dynamically, and once
a stream is shut down, a new one cannot be created in the same
sparkcontext. So we cannot create a service that would create and manage
multiple streams - the same way that is possible with batch jobs.

On Mon, Mar 2, 2015 at 2:52 PM, Sean Owen <sowen@cloudera.com> wrote:

> I think everything there is to know about it is on JIRA; I don't think
> that's being worked on.
>
> On Mon, Mar 2, 2015 at 2:50 PM, Tamas Jambor <jamborta@gmail.com> wrote:
> > I have seen there is a card (SPARK-2243) to enable that. Is that still
> going
> > ahead?
> >
> > On Mon, Mar 2, 2015 at 2:46 PM, Sean Owen <sowen@cloudera.com> wrote:
> >>
> >> It is still not something you're supposed to do; in fact there is a
> >> setting (disabled by default) that throws an exception if you try to
> >> make multiple contexts.
> >>
> >> On Mon, Mar 2, 2015 at 2:43 PM, jamborta <jamborta@gmail.com> wrote:
> >> > hi all,
> >> >
> >> > what is the current status and direction on enabling multiple
> >> > sparkcontexts
> >> > and streamingcontext? I have seen a few issues open on JIRA, which
> seem
> >> > to
> >> > be there for quite a while.
> >> >
> >> > thanks,
> >> >
> >> >
> >> >
> >> > --
> >> > View this message in context:
> >> >
> http://apache-spark-user-list.1001560.n3.nabble.com/multiple-sparkcontexts-and-streamingcontexts-tp21876.html
> >> > Sent from the Apache Spark User List mailing list archive at
> Nabble.com.
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> >> > For additional commands, e-mail: user-help@spark.apache.org
> >> >
> >
> >
>

Mime
View raw message