spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Hadoop configuration for checkpointing
Date Wed, 05 Nov 2014 03:40:34 GMT
Opened
https://issues.apache.org/jira/browse/SPARK-4229

Sent a PR
https://github.com/apache/spark/pull/3102

On Tue, Nov 4, 2014 at 11:48 AM, Marcelo Vanzin <vanzin@cloudera.com> wrote:

> On Tue, Nov 4, 2014 at 9:34 AM, Cody Koeninger <cody@koeninger.org> wrote:
> > 2.  Is there a reason StreamingContext.getOrCreate defaults to a blank
> > hadoop configuration rather than
> > org.apache.spark.deploy.SparkHadoopUtil.get.conf,
> > which would pull values from spark config?
>
> This is probably something I overlooked when I changed the rest of the
> code to use SparkHadoopUtil.get.conf. Feel free to send a PR for it,
> we should get it into 1.2 so that all code creates configuration
> objects in a consistent way.
>
> --
> Marcelo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message