spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: [VOTE] Release Apache Spark 1.4.1
Date Thu, 25 Jun 2015 09:38:36 GMT
That makes sense to me -- there's an urgent fix to get out. I missed
that part. Not that it really matters but was that expressed

I know we tend to start the RC process even when a few more changes
are still in progress, to get a first wave or two of testing done
early, knowing that the RC won't be the final one. It makes sense for
some issues for X to be open when an RC is cut, if they are actually
truly intended for X.

44 seems like a lot, and I don't think it's good practice just because
that's how it's happened before. It looks like half of them weren't
actually important for 1.4.x as we're now down to 21. I don't disagree
with the idea that only "most" of the issues targeted for version X
will be in version X; the target expresses a "stretch goal". Given the
fast pace of change that's probably the only practical view.

I think we're just missing a step then: before RC of X, ask people to
review and update the target of JIRAs for X? In this case, it was a
good point to untarget stuff from 1.4.x entirely; I suspect everything
else should then be targeted at 1.4.2 by default with the exception of
a handful that people really do intend to work in for 1.4.1 before its
final release.

I know it sounds like pencil-pushing, but it's a cheap way to bring
some additional focus to release planning. RC time has felt like a
last-call to *begin* changes ad-hoc when it would go faster if it were
more intentional and constrained. Meaning faster RCs, meaning getting
back to a 3-month release cycle or less, and meaning less rush to push
stuff into a .0 release and less frequent need for a maintenance .1

So what happens if all 1.4.1-targeted JIRAs are targeted to 1.4.2?
would that miss something that is definitely being worked on for

On Wed, Jun 24, 2015 at 6:56 PM, Patrick Wendell <> wrote:
> Hey Sean,
> This is being shipped now because there is a severe bug in 1.4.0 that
> can cause data corruption for Parquet users.
> There are no blockers targeted for 1.4.1 - so I don't see that JIRA is
> inconsistent with shipping a release now. The goal of having every
> single targeted JIRA cleared by the time we start voting, I don't
> think there is broad consensus and cultural adoption of that principle
> yet. So I do not take it as a signal this release is premature (the
> story has been the same for every previous release we've ever done).
> The fact that we hit 90/124 of issues targeted at this release means
> we are targeting such that we get around 70% of issues merged. That
> actually doesn't seem so bad to me since there is some uncertainty in
> the process. B
> - Patrick
> On Wed, Jun 24, 2015 at 1:54 AM, Sean Owen <> wrote:
>> There are 44 issues still targeted for 1.4.1. None are Blockers; 12
>> are Critical. ~80% were opened and/or set by committers. Compare with
>> 90 issues resolved for 1.4.1.
>> I'm concerned that committers are targeting lots more for a release
>> even in the short term than realistically can go in. On its face, it
>> suggests that an RC is premature. Why is 1.4.1 being put forth for
>> release now? It seems like people are saying they want a fair bit more
>> time to work on 1.4.1.
>> I suspect that in fact people would rather untarget / slip (again)
>> these JIRAs, but it calls into question again how the targeting is
>> consistently off by this much.
>> What unresolved JIRAs targeted for 1.4.1 are *really* still open for
>> 1.4.1? like, what would go badly if all 32 non-Critical JIRAs were
>> untargeted now? is the reality that there are a handful of items to
>> get in before the final release, and those are hopefully the ~12
>> critical ones? How about some review of that before we ask people to
>> seriously test these bits?
>> On Wed, Jun 24, 2015 at 8:37 AM, Patrick Wendell <> wrote:
>>> Please vote on releasing the following candidate as Apache Spark version 1.4.1!
>>> This release fixes a handful of known issues in Spark 1.4.0, listed here:
>>> The tag to be voted on is v1.4.1-rc1 (commit 60e08e5):
>>> 60e08e50751fe3929156de956d62faea79f5b801
>>> The release files, including signatures, digests, etc. can be found at:
>>> Release artifacts are signed with the following key:
>>> The staging repository for this release can be found at:
>>> [published as version: 1.4.1]
>>> [published as version: 1.4.1-rc1]
>>> The documentation corresponding to this release can be found at:
>>> Please vote on releasing this package as Apache Spark 1.4.1!
>>> The vote is open until Saturday, June 27, at 06:32 UTC and passes
>>> if a majority of at least 3 +1 PMC votes are cast.
>>> [ ] +1 Release this package as Apache Spark 1.4.1
>>> [ ] -1 Do not release this package because ...
>>> To learn more about Apache Spark, please see
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message