spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: Running K8s integration tests for changes in core?
Date Thu, 20 Aug 2020 21:28:40 GMT
Sounds good, thanks for the heads up. I hope you get some time to relax :)

On Thu, Aug 20, 2020 at 2:26 PM shane knapp ☠ <sknapp@berkeley.edu> wrote:

> fyi, i won't be making this change until the 1st week of september.  i'll
> be out, off the grid all next week!  :)
>
> i will send an announcement out tomorrow on how to contact my team here @
> uc berkeley if jenkins goes down.
>
> shane
>
> On Thu, Aug 20, 2020 at 4:40 AM Prashant Sharma <scrapcodes@gmail.com>
> wrote:
>
>> Another option is, if we could have something like "presubmit" PR build.
>> In other words, running the entire 4 H + K8s integration on each commit
>> pushed is too much at the same time and there are chances that one thing
>> can inadvertently affect other components(as you just said).
>>
>> A presubmit(which includes K8s integration tests) build will be run, once
>> the PR receives LGTM from "Approved reviewers". This is one criteria that
>> comes to my mind, others may have better suggestions.
>>
>> On Thu, Aug 20, 2020 at 12:25 AM shane knapp ☠ <sknapp@berkeley.edu>
>> wrote:
>>
>>> we'll be gated by the number of ubuntu workers w/minikube and docker,
>>> but it shouldn't be too bad as the full integration test takes ~45m, vs 4+
>>> hrs for the regular PRB.
>>>
>>> i can enable this in about 1m of time if the consensus is for us to want
>>> this.
>>>
>>> On Wed, Aug 19, 2020 at 11:37 AM Holden Karau <holden@pigscanfly.ca>
>>> wrote:
>>>
>>>> Sounds good. In the meantime would folks committing things in core run
>>>> the K8s PRB or run it locally? A second change this morning was committed
>>>> that broke the K8s PR tests.
>>>>
>>>> On Tue, Aug 18, 2020 at 9:53 PM Prashant Sharma <scrapcodes@gmail.com>
>>>> wrote:
>>>>
>>>>> +1, we should enable.
>>>>>
>>>>> On Wed, Aug 19, 2020 at 9:18 AM Holden Karau <holden@pigscanfly.ca>
>>>>> wrote:
>>>>>
>>>>>> Hi Dev Folks,
>>>>>>
>>>>>> I was wondering how people feel about enabling the K8s PRB
>>>>>> automatically for all core changes? Sometimes I forget that a change
might
>>>>>> impact one of the K8s integration tests since a bunch of them look
at log
>>>>>> messages. Would folks be OK with turning on the K8s integration PRB
for all
>>>>>> core changes as well as K8s changes?
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Holden :)
>>>>>>
>>>>>> --
>>>>>> Twitter: https://twitter.com/holdenkarau
>>>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>>>> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
>>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Twitter: https://twitter.com/holdenkarau
>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>>
>>>
>>> --
>>> Shane Knapp
>>> Computer Guy / Voice of Reason
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>>
>>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Mime
View raw message