spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: Spark 1.6.1 Hadoop 2.6 package on S3 corrupt?
Date Mon, 04 Apr 2016 21:39:53 GMT
An additional note: The Spark packages being served off of CloudFront (i.e.
the “direct download” option on spark.apache.org) are also corrupt.

Btw what’s the correct way to verify the SHA of a Spark package? I’ve tried
a few commands on working packages downloaded from Apache mirrors, but I
can’t seem to reproduce the published SHA for spark-1.6.1-bin-hadoop2.6.tgz
<http://www.apache.org/dist/spark/spark-1.6.1/spark-1.6.1-bin-hadoop2.6.tgz.sha>
.
​

On Mon, Apr 4, 2016 at 11:45 AM Ted Yu <yuzhihong@gmail.com> wrote:

> Maybe temporarily take out the artifacts on S3 before the root cause is
> found.
>
> On Thu, Mar 24, 2016 at 7:25 AM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> Just checking in on this again as the builds on S3 are still broken. :/
>>
>> Could it have something to do with us moving release-build.sh
>> <https://github.com/apache/spark/commits/master/dev/create-release/release-build.sh>
>> ?
>> ​
>>
>> On Mon, Mar 21, 2016 at 1:43 PM Nicholas Chammas <
>> nicholas.chammas@gmail.com> wrote:
>>
>>> Is someone going to retry fixing these packages? It's still a problem.
>>>
>>> Also, it would be good to understand why this is happening.
>>>
>>> On Fri, Mar 18, 2016 at 6:49 PM Jakob Odersky <jakob@odersky.com> wrote:
>>>
>>>> I just realized you're using a different download site. Sorry for the
>>>> confusion, the link I get for a direct download of Spark 1.6.1 /
>>>> Hadoop 2.6 is
>>>> http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz
>>>>
>>>> On Fri, Mar 18, 2016 at 3:20 PM, Nicholas Chammas
>>>> <nicholas.chammas@gmail.com> wrote:
>>>> > I just retried the Spark 1.6.1 / Hadoop 2.6 download and got a
>>>> corrupt ZIP
>>>> > file.
>>>> >
>>>> > Jakob, are you sure the ZIP unpacks correctly for you? Is it the same
>>>> Spark
>>>> > 1.6.1/Hadoop 2.6 package you had a success with?
>>>> >
>>>> > On Fri, Mar 18, 2016 at 6:11 PM Jakob Odersky <jakob@odersky.com>
>>>> wrote:
>>>> >>
>>>> >> I just experienced the issue, however retrying the download a second
>>>> >> time worked. Could it be that there is some load balancer/cache
in
>>>> >> front of the archive and some nodes still serve the corrupt packages?
>>>> >>
>>>> >> On Fri, Mar 18, 2016 at 8:00 AM, Nicholas Chammas
>>>> >> <nicholas.chammas@gmail.com> wrote:
>>>> >> > I'm seeing the same. :(
>>>> >> >
>>>> >> > On Fri, Mar 18, 2016 at 10:57 AM Ted Yu <yuzhihong@gmail.com>
>>>> wrote:
>>>> >> >>
>>>> >> >> I tried again this morning :
>>>> >> >>
>>>> >> >> $ wget
>>>> >> >>
>>>> >> >>
>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >> --2016-03-18 07:55:30--
>>>> >> >>
>>>> >> >>
>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >> Resolving s3.amazonaws.com... 54.231.19.163
>>>> >> >> ...
>>>> >> >> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >>
>>>> >> >> gzip: stdin: unexpected end of file
>>>> >> >> tar: Unexpected EOF in archive
>>>> >> >> tar: Unexpected EOF in archive
>>>> >> >> tar: Error is not recoverable: exiting now
>>>> >> >>
>>>> >> >> On Thu, Mar 17, 2016 at 8:57 AM, Michael Armbrust
>>>> >> >> <michael@databricks.com>
>>>> >> >> wrote:
>>>> >> >>>
>>>> >> >>> Patrick reuploaded the artifacts, so it should be fixed
now.
>>>> >> >>>
>>>> >> >>> On Mar 16, 2016 5:48 PM, "Nicholas Chammas"
>>>> >> >>> <nicholas.chammas@gmail.com>
>>>> >> >>> wrote:
>>>> >> >>>>
>>>> >> >>>> Looks like the other packages may also be corrupt.
I’m getting
>>>> the
>>>> >> >>>> same
>>>> >> >>>> error for the Spark 1.6.1 / Hadoop 2.4 package.
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.4.tgz
>>>> >> >>>>
>>>> >> >>>> Nick
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> On Wed, Mar 16, 2016 at 8:28 PM Ted Yu <yuzhihong@gmail.com>
>>>> wrote:
>>>> >> >>>>>
>>>> >> >>>>> On Linux, I got:
>>>> >> >>>>>
>>>> >> >>>>> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >>>>>
>>>> >> >>>>> gzip: stdin: unexpected end of file
>>>> >> >>>>> tar: Unexpected EOF in archive
>>>> >> >>>>> tar: Unexpected EOF in archive
>>>> >> >>>>> tar: Error is not recoverable: exiting now
>>>> >> >>>>>
>>>> >> >>>>> On Wed, Mar 16, 2016 at 5:15 PM, Nicholas Chammas
>>>> >> >>>>> <nicholas.chammas@gmail.com> wrote:
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >>>>>>
>>>> >> >>>>>> Does anyone else have trouble unzipping
this? How did this
>>>> happen?
>>>> >> >>>>>>
>>>> >> >>>>>> What I get is:
>>>> >> >>>>>>
>>>> >> >>>>>> $ gzip -t spark-1.6.1-bin-hadoop2.6.tgz
>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: unexpected
end of file
>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: uncompress
failed
>>>> >> >>>>>>
>>>> >> >>>>>> Seems like a strange type of problem to
come across.
>>>> >> >>>>>>
>>>> >> >>>>>> Nick
>>>> >> >>>>>
>>>> >> >>>>>
>>>> >> >>
>>>> >> >
>>>>
>>>
>

Mime
View raw message