spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Sakellis <kos...@cloudera.com>
Subject Re: [VOTE] Apache Spark 2.2.0 (RC2)
Date Wed, 31 May 2017 18:58:46 GMT
Hey Michael,

There is a discussion on TIMESTAMP semantics going on the thread "SQL
TIMESTAMP semantics vs. SPARK-18350" which might impact Spark 2.2. Should
we make a decision there before voting on the next RC for Spark 2.2?

Thanks,
Kostas

On Tue, May 30, 2017 at 12:09 PM, Michael Armbrust <michael@databricks.com>
wrote:

> Last call, anything else important in-flight for 2.2?
>
> On Thu, May 25, 2017 at 10:56 AM, Michael Allman <michael@videoamp.com>
> wrote:
>
>> PR is here: https://github.com/apache/spark/pull/18112
>>
>>
>> On May 25, 2017, at 10:28 AM, Michael Allman <michael@videoamp.com>
>> wrote:
>>
>> Michael,
>>
>> If you haven't started cutting the new RC, I'm working on a documentation
>> PR right now I'm hoping we can get into Spark 2.2 as a migration note, even
>> if it's just a mention: https://issues.apache.org/jira/browse/SPARK-20888
>> .
>>
>> Michael
>>
>>
>> On May 22, 2017, at 11:39 AM, Michael Armbrust <michael@databricks.com>
>> wrote:
>>
>> I'm waiting for SPARK-20814
>> <https://issues.apache.org/jira/browse/SPARK-20814> at Marcelo's
>> request and I'd also like to include SPARK-20844
>> <https://issues.apache.org/jira/browse/SPARK-20844>.  I think we should
>> be able to cut another RC midweek.
>>
>> On Fri, May 19, 2017 at 11:53 AM, Nick Pentreath <
>> nick.pentreath@gmail.com> wrote:
>>
>>> All the outstanding ML QA doc and user guide items are done for 2.2 so
>>> from that side we should be good to cut another RC :)
>>>
>>>
>>> On Thu, 18 May 2017 at 00:18 Russell Spitzer <russell.spitzer@gmail.com>
>>> wrote:
>>>
>>>> Seeing an issue with the DataScanExec and some of our integration tests
>>>> for the SCC. Running dataframe read and writes from the shell seems fine
>>>> but the Redaction code seems to get a "None" when doing
>>>> SparkSession.getActiveSession.get in our integration tests. I'm not
>>>> sure why but i'll dig into this later if I get a chance.
>>>>
>>>> Example Failed Test
>>>> https://github.com/datastax/spark-cassandra-connector/blob/v
>>>> 2.0.1/spark-cassandra-connector/src/it/scala/com/datastax/sp
>>>> ark/connector/sql/CassandraSQLSpec.scala#L311
>>>>
>>>> ```[info]   org.apache.spark.SparkException: Job aborted due to stage
>>>> failure: Task serialization failed: java.util.NoSuchElementException:
>>>> None.get
>>>> [info] java.util.NoSuchElementException: None.get
>>>> [info] at scala.None$.get(Option.scala:347)
>>>> [info] at scala.None$.get(Option.scala:345)
>>>> [info] at org.apache.spark.sql.execution.DataSourceScanExec$class.org$
>>>> apache$spark$sql$execution$DataSourceScanExec$$redact(DataSo
>>>> urceScanExec.scala:70)
>>>> [info] at org.apache.spark.sql.execution.DataSourceScanExec$$anonfun$4
>>>> .apply(DataSourceScanExec.scala:54)
>>>> [info] at org.apache.spark.sql.execution.DataSourceScanExec$$anonfun$4
>>>> .apply(DataSourceScanExec.scala:52)
>>>> ```
>>>>
>>>> Again this only seems to repo in our IT suite so i'm not sure if this
>>>> is a real issue.
>>>>
>>>>
>>>> On Tue, May 16, 2017 at 1:40 PM Joseph Bradley <joseph@databricks.com>
>>>> wrote:
>>>>
>>>>> All of the ML/Graph/SparkR QA blocker JIRAs have been resolved.
>>>>> Thanks everyone who helped out on those!
>>>>>
>>>>> We still have open ML/Graph/SparkR JIRAs targeted at 2.2, but they are
>>>>> essentially all for documentation.
>>>>>
>>>>> Joseph
>>>>>
>>>>> On Thu, May 11, 2017 at 3:08 PM, Marcelo Vanzin <vanzin@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Since you'll be creating a new RC, I'd wait until SPARK-20666 is
>>>>>> fixed, since the change that caused it is in branch-2.2. Probably
a
>>>>>> good idea to raise it to blocker at this point.
>>>>>>
>>>>>> On Thu, May 11, 2017 at 2:59 PM, Michael Armbrust
>>>>>> <michael@databricks.com> wrote:
>>>>>> > I'm going to -1 given the outstanding issues and lack of +1s.
 I'll
>>>>>> create
>>>>>> > another RC once ML has had time to take care of the more critical
>>>>>> problems.
>>>>>> > In the meantime please keep testing this release!
>>>>>> >
>>>>>> > On Tue, May 9, 2017 at 2:00 AM, Kazuaki Ishizaki <
>>>>>> ISHIZAKI@jp.ibm.com>
>>>>>> > wrote:
>>>>>> >>
>>>>>> >> +1 (non-binding)
>>>>>> >>
>>>>>> >> I tested it on Ubuntu 16.04 and OpenJDK8 on ppc64le. All
of the
>>>>>> tests for
>>>>>> >> core have passed.
>>>>>> >>
>>>>>> >> $ java -version
>>>>>> >> openjdk version "1.8.0_111"
>>>>>> >> OpenJDK Runtime Environment (build
>>>>>> >> 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14)
>>>>>> >> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>>>>>> >> $ build/mvn -DskipTests -Phive -Phive-thriftserver -Pyarn
>>>>>> -Phadoop-2.7
>>>>>> >> package install
>>>>>> >> $ build/mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.7
test
>>>>>> -pl core
>>>>>> >> ...
>>>>>> >> Run completed in 15 minutes, 12 seconds.
>>>>>> >> Total number of tests run: 1940
>>>>>> >> Suites: completed 206, aborted 0
>>>>>> >> Tests: succeeded 1940, failed 0, canceled 4, ignored 8,
pending 0
>>>>>> >> All tests passed.
>>>>>> >> [INFO]
>>>>>> >> ------------------------------------------------------------
>>>>>> ------------
>>>>>> >> [INFO] BUILD SUCCESS
>>>>>> >> [INFO]
>>>>>> >> ------------------------------------------------------------
>>>>>> ------------
>>>>>> >> [INFO] Total time: 16:51 min
>>>>>> >> [INFO] Finished at: 2017-05-09T17:51:04+09:00
>>>>>> >> [INFO] Final Memory: 53M/514M
>>>>>> >> [INFO]
>>>>>> >> ------------------------------------------------------------
>>>>>> ------------
>>>>>> >> [WARNING] The requested profile "hive" could not be activated
>>>>>> because it
>>>>>> >> does not exist.
>>>>>> >>
>>>>>> >>
>>>>>> >> Kazuaki Ishizaki,
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> From:        Michael Armbrust <michael@databricks.com>
>>>>>> >> To:        "dev@spark.apache.org" <dev@spark.apache.org>
>>>>>> >> Date:        2017/05/05 02:08
>>>>>> >> Subject:        [VOTE] Apache Spark 2.2.0 (RC2)
>>>>>> >> ________________________________
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> Please vote on releasing the following candidate as Apache
Spark
>>>>>> version
>>>>>> >> 2.2.0. The vote is open until Tues, May 9th, 2017 at 12:00
PST and
>>>>>> passes if
>>>>>> >> a majority of at least 3 +1 PMC votes are cast.
>>>>>> >>
>>>>>> >> [ ] +1 Release this package as Apache Spark 2.2.0
>>>>>> >> [ ] -1 Do not release this package because ...
>>>>>> >>
>>>>>> >>
>>>>>> >> To learn more about Apache Spark, please see
>>>>>> http://spark.apache.org/
>>>>>> >>
>>>>>> >> The tag to be voted on is v2.2.0-rc2
>>>>>> >> (1d4017b44d5e6ad156abeaae6371747f111dd1f9)
>>>>>> >>
>>>>>> >> List of JIRA tickets resolved can be found with this filter.
>>>>>> >>
>>>>>> >> The release files, including signatures, digests, etc. can
be
>>>>>> found at:
>>>>>> >> http://home.apache.org/~pwendell/spark-releases/spark-2.2.0-
>>>>>> rc2-bin/
>>>>>> >>
>>>>>> >> Release artifacts are signed with the following key:
>>>>>> >> https://people.apache.org/keys/committer/pwendell.asc
>>>>>> >>
>>>>>> >> The staging repository for this release can be found at:
>>>>>> >> https://repository.apache.org/content/repositories/orgapache
>>>>>> spark-1236/
>>>>>> >>
>>>>>> >> The documentation corresponding to this release can be found
at:
>>>>>> >> http://people.apache.org/~pwendell/spark-releases/spark-2.2.
>>>>>> 0-rc2-docs/
>>>>>> >>
>>>>>> >>
>>>>>> >> FAQ
>>>>>> >>
>>>>>> >> How can I help test this release?
>>>>>> >>
>>>>>> >> If you are a Spark user, you can help us test this release
by
>>>>>> taking an
>>>>>> >> existing Spark workload and running on this release candidate,
then
>>>>>> >> reporting any regressions.
>>>>>> >>
>>>>>> >> What should happen to JIRA tickets still targeting 2.2.0?
>>>>>> >>
>>>>>> >> Committers should look at those and triage. Extremely important
>>>>>> bug fixes,
>>>>>> >> documentation, and API tweaks that impact compatibility
should be
>>>>>> worked on
>>>>>> >> immediately. Everything else please retarget to 2.3.0 or
2.2.1.
>>>>>> >>
>>>>>> >> But my bug isn't fixed!??!
>>>>>> >>
>>>>>> >> In order to make timely releases, we will typically not
hold the
>>>>>> release
>>>>>> >> unless the bug in question is a regression from 2.1.1.
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Marcelo
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Joseph Bradley
>>>>> Software Engineer - Machine Learning
>>>>> Databricks, Inc.
>>>>> [image: http://databricks.com] <http://databricks.com/>
>>>>>
>>>>
>>
>>
>>
>

Mime
View raw message