spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <>
Subject Re: Any ideas on SPARK-1021?
Date Tue, 13 May 2014 03:42:49 GMT
This is the issue where .sortByKey() launches a cluster job when it
shouldn't because it's a transformation not an action.

I'd appreciate a fix too but don't currently have any thoughts on how to
proceed forward.


On Thu, May 8, 2014 at 2:16 PM, Mark Hamstra <>wrote:

> I'm trying to decide whether attacking the underlying issue of
> RangePartitioner running eager jobs in rangeBounds (i.e. SPARK-1021) is a
> better option than a messy workaround for some async job-handling stuff
> that I am working on.  It looks like there have been a couple of aborted
> attempts to solve the problem, but no solution or clear path to one at this
> point.
> Has anybody made any further progress or does anyone have any new ideas on
> how to proceed?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message