spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <>
Subject Re: Setup Remote HDFS for Spark
Date Fri, 21 Nov 2014 14:40:27 GMT
Having them on the same network will give you better performance. What
problems are you facing? Can you elaborate more about the versions? (Spark
and hadoop), Only problem that could happen is the OS's OOM Killer might be
killing your HDFS processes due to low  memory and such. But you can check
the logs to get more clear picture about whats happening.

Best Regards

On Fri, Nov 21, 2014 at 7:58 PM, EH <> wrote:

> Hi,
> Are there any way that I can setup a remote HDFS for Spark (more specific,
> for Spark Streaming checkpoints)?  The reason I'm asking is that our Spark
> and HDFS do not run on the same machines.  I've been looked around but
> still
> no clue so far.
> Thanks,
> EH
> --
> View this message in context:
> Sent from the Apache Spark User List mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message