spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <and...@andrewash.com>
Subject Re: Any ideas on SPARK-1021?
Date Tue, 13 May 2014 03:42:49 GMT
This is the issue where .sortByKey() launches a cluster job when it
shouldn't because it's a transformation not an action.

https://issues.apache.org/jira/browse/SPARK-1021

I'd appreciate a fix too but don't currently have any thoughts on how to
proceed forward.

Andrew


On Thu, May 8, 2014 at 2:16 PM, Mark Hamstra <mark@clearstorydata.com>wrote:

> I'm trying to decide whether attacking the underlying issue of
> RangePartitioner running eager jobs in rangeBounds (i.e. SPARK-1021) is a
> better option than a messy workaround for some async job-handling stuff
> that I am working on.  It looks like there have been a couple of aborted
> attempts to solve the problem, but no solution or clear path to one at this
> point.
>
> Has anybody made any further progress or does anyone have any new ideas on
> how to proceed?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message