An additional note: The Spark packages being served off of CloudFront (i.e. the “direct download” option on spark.apache.org) are also corrupt.

Btw what’s the correct way to verify the SHA of a Spark package? I’ve tried a few commands on working packages downloaded from Apache mirrors, but I can’t seem to reproduce the published SHA for spark-1.6.1-bin-hadoop2.6.tgz.


On Mon, Apr 4, 2016 at 11:45 AM Ted Yu <yuzhihong@gmail.com> wrote:
Maybe temporarily take out the artifacts on S3 before the root cause is found.

On Thu, Mar 24, 2016 at 7:25 AM, Nicholas Chammas <nicholas.chammas@gmail.com> wrote:

Just checking in on this again as the builds on S3 are still broken. :/

Could it have something to do with us moving release-build.sh?


On Mon, Mar 21, 2016 at 1:43 PM Nicholas Chammas <nicholas.chammas@gmail.com> wrote:
Is someone going to retry fixing these packages? It's still a problem.

Also, it would be good to understand why this is happening.

On Fri, Mar 18, 2016 at 6:49 PM Jakob Odersky <jakob@odersky.com> wrote:
I just realized you're using a different download site. Sorry for the
confusion, the link I get for a direct download of Spark 1.6.1 /
Hadoop 2.6 is http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz

On Fri, Mar 18, 2016 at 3:20 PM, Nicholas Chammas
<nicholas.chammas@gmail.com> wrote:
> I just retried the Spark 1.6.1 / Hadoop 2.6 download and got a corrupt ZIP
> file.
>
> Jakob, are you sure the ZIP unpacks correctly for you? Is it the same Spark
> 1.6.1/Hadoop 2.6 package you had a success with?
>
> On Fri, Mar 18, 2016 at 6:11 PM Jakob Odersky <jakob@odersky.com> wrote:
>>
>> I just experienced the issue, however retrying the download a second
>> time worked. Could it be that there is some load balancer/cache in
>> front of the archive and some nodes still serve the corrupt packages?
>>
>> On Fri, Mar 18, 2016 at 8:00 AM, Nicholas Chammas
>> <nicholas.chammas@gmail.com> wrote:
>> > I'm seeing the same. :(
>> >
>> > On Fri, Mar 18, 2016 at 10:57 AM Ted Yu <yuzhihong@gmail.com> wrote:
>> >>
>> >> I tried again this morning :
>> >>
>> >> $ wget
>> >>
>> >> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>> >> --2016-03-18 07:55:30--
>> >>
>> >> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>> >> Resolving s3.amazonaws.com... 54.231.19.163
>> >> ...
>> >> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz
>> >>
>> >> gzip: stdin: unexpected end of file
>> >> tar: Unexpected EOF in archive
>> >> tar: Unexpected EOF in archive
>> >> tar: Error is not recoverable: exiting now
>> >>
>> >> On Thu, Mar 17, 2016 at 8:57 AM, Michael Armbrust
>> >> <michael@databricks.com>
>> >> wrote:
>> >>>
>> >>> Patrick reuploaded the artifacts, so it should be fixed now.
>> >>>
>> >>> On Mar 16, 2016 5:48 PM, "Nicholas Chammas"
>> >>> <nicholas.chammas@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> Looks like the other packages may also be corrupt. I’m getting the
>> >>>> same
>> >>>> error for the Spark 1.6.1 / Hadoop 2.4 package.
>> >>>>
>> >>>>
>> >>>>
>> >>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.4.tgz
>> >>>>
>> >>>> Nick
>> >>>>
>> >>>>
>> >>>> On Wed, Mar 16, 2016 at 8:28 PM Ted Yu <yuzhihong@gmail.com> wrote:
>> >>>>>
>> >>>>> On Linux, I got:
>> >>>>>
>> >>>>> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz
>> >>>>>
>> >>>>> gzip: stdin: unexpected end of file
>> >>>>> tar: Unexpected EOF in archive
>> >>>>> tar: Unexpected EOF in archive
>> >>>>> tar: Error is not recoverable: exiting now
>> >>>>>
>> >>>>> On Wed, Mar 16, 2016 at 5:15 PM, Nicholas Chammas
>> >>>>> <nicholas.chammas@gmail.com> wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz
>> >>>>>>
>> >>>>>> Does anyone else have trouble unzipping this? How did this happen?
>> >>>>>>
>> >>>>>> What I get is:
>> >>>>>>
>> >>>>>> $ gzip -t spark-1.6.1-bin-hadoop2.6.tgz
>> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: unexpected end of file
>> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: uncompress failed
>> >>>>>>
>> >>>>>> Seems like a strange type of problem to come across.
>> >>>>>>
>> >>>>>> Nick
>> >>>>>
>> >>>>>
>> >>
>> >