spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: [VOTE] SPARK 2.4.0 (RC4)
Date Tue, 23 Oct 2018 16:03:07 GMT
I am searching and checking some PRs or JIRAs that state regression. Let me
leave a link - it might be good to double check
https://github.com/apache/spark/pull/22514 as well.

2018년 10월 23일 (화) 오후 11:58, Stavros Kontopoulos <
stavros.kontopoulos@lightbend.com>님이 작성:

> Sean,
>
> I will try it against 2.12 shortly.
>
> You're saying someone would have to first build a k8s distro from source
>> too?
>
>
> Ok I missed the error one line above, before the distro error there is
> another one:
>
> fatal: not a git repository (or any of the parent directories): .git
>
>
> So that seems to come from here
> <https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/scripts/setup-integration-test-env.sh#L19>.
> It seems that the test root is not set up correctly. It should be the top
> git dir from which you built Spark.
>
> Now regarding the distro thing. dev-run-integration-tests.sh should run
> from within the cloned project after the distro is built. The distro is
> required
> <https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/scripts/setup-integration-test-env.sh#L61>
> , it should fail otherwise.
>
> Integration tests run the setup-integration-test-env.sh script. dev-run-integration-tests.sh
> calls mvn
> <https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh#L106>
which
> in turn executes that setup script
> <https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/pom.xml#L80>
> .
>
> How do you run the tests?
>
> Stavros
>
> On Tue, Oct 23, 2018 at 3:01 PM, Sean Owen <srowen@gmail.com> wrote:
>
>> No, because the docs are built into the release too and released to
>> the site too from the released artifact.
>> As a practical matter, I think these docs are not critical for
>> release, and can follow in a maintenance release. I'd retarget to
>> 2.4.1 or untarget.
>> I do know at times a release's docs have been edited after the fact,
>> but that's bad form. We'd not go change a class in the release after
>> it was released and call it the same release.
>>
>> I'd still like some confirmation that someone can build and pass tests
>> with -Pkubernetes, maybe? It actually all passed with the 2.11 build.
>> I don't think it's a 2.12 incompatibility, but rather than the K8S
>> tests maybe don't quite work with the 2.12 build artifact naming. Or
>> else something to do with my env.
>>
>> On Mon, Oct 22, 2018 at 9:08 PM Wenchen Fan <cloud0fan@gmail.com> wrote:
>> >
>> > Regarding the doc tickets, I vaguely remember that we can merge doc PRs
>> after release and publish doc to spark website later. Can anyone confirm?
>> >
>> > On Tue, Oct 23, 2018 at 8:30 AM Sean Owen <srowen@gmail.com> wrote:
>> >>
>> >> This is what I got from a straightforward build of the source distro
>> >> here ... really, ideally, it builds as-is from source. You're saying
>> >> someone would have to first build a k8s distro from source too?
>> >> It's not a 'must' that this be automatic but nothing else fails out of
>> the box.
>> >> I feel like I might be misunderstanding the setup here.
>> >> On Mon, Oct 22, 2018 at 7:25 PM Stavros Kontopoulos
>> >> <stavros.kontopoulos@lightbend.com> wrote:
>>
>
>
>
> --
> Stavros Kontopoulos
>
> *Senior Software Engineer*
> *Lightbend, Inc.*
>
> *p:  +30 6977967274 <%2B1%20650%20678%200020>*
> *e: stavros.kontopoulos@lightbend.com* <dave.martin@lightbend.com>
>
>
>

Mime
View raw message