whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: Problem using blobstore
Date Tue, 02 Aug 2011 20:17:57 GMT
Looks good to me but I'm not convinced that removing the file:// prefix is
the right way of handling paths on Windows.

I was also suspecting an intermittent AWS failure. I'm glad they managed to
fix it.

Could you create a patch from that pull request and attach to WHIRR-349 so
that we can also handle the multipart uploads before committing?

Thanks,

-- Andrei Savu / andreisavu.ro

On Mon, Aug 1, 2011 at 11:37 PM, Karel Vervaeke <karel@outerthought.org>wrote:

> Hi there.
>
> Yesterday I have digging into this problem. Before the end of the day
> my uploads to eu-west-1 started working intermittently, and by the end
> of the day I couldn't reproduce the problem anymore (without any
> changes to the code).
>
> I guess I don't have enough cloud-karma.
>
> For what it's worth I have sent a pull request with a partial request
> for WHIRR-349
> What I've done is just remove the prefix 'file://' from the url. I
> don't know if that's correct way to do it for Windows-style paths.
> It also includes a testcase for uploading slightly larger blobs (a 1MB
> random alphanumeric string).
> I haven't gone into the multiparts uploads.
>
> Regards,
> Karel
>
> On Fri, Jul 29, 2011 at 11:53 AM, Karel Vervaeke <karel@outerthought.org>
> wrote:
> > I just got around to testing all 4 combinations:
> >
> > jclouds 1.0.0 + eu-west-1: doesn't work
> > jclouds 1.0.0 + us-east-1: works
> > jclouds 1.1.0-SNAPSHOT + eu-west-1: doesn't work
> > jclouds 1.1.0-SNAPSHOT + us-east-1: works
> >
> > Would you be willing to try using the s3 blobstore on eu-west-1 so I
> > know if it's just me or not?
> >
> > Regards,
> > Karel
> >
> > On Fri, Jul 29, 2011 at 10:49 AM, Karel Vervaeke <karel@outerthought.org>
> wrote:
> >> with jclouds 1.0.0 it failed with eu-west-1 (several attempts)
> >> with jclouds 1.0.0 it worked with us-east-1 on the first try.
> >>
> >> I still have to try with jclouds 1.1.0-SNAPSHOT on us-east-1, but if
> that works,
> >> the problem is related to eu-west-1.
> >>
> >> Regards,
> >> Karel
> >>
> >> On Fri, Jul 29, 2011 at 9:49 AM, Karel Vervaeke <karel@outerthought.org>
> wrote:
> >>> Ah, Bruno just alerted me that I need to mention that I am using his
> >>> (github) whirr fork, which is using a jclouds 1.1 snapshot.
> >>>
> >>> I can fall back to using jclouds 1.0 for a second (with the limitation
> >>> that I won't be able to use eu-west-1. Just give me a second while I
> >>> try that. (And in the mean time I'll open the upload-retry issue)
> >>>
> >>> Regards,
> >>> Karel
> >>>
> >>> On Fri, Jul 29, 2011 at 9:28 AM, Andrei Savu <savu.andrei@gmail.com>
> wrote:
> >>>> I've just took a quick look at the AWS service status dashboard and
> >>>> everything seems to be working there.
> >>>>
> >>>> The error you are seeing when trying to upload a larger file is
> >>>> something that I believe we should handle by retrying the upload from
> >>>> scratch. It would be great if you could open an issue with this
> >>>> suggestion.
> >>>>
> >>>> I have no idea why the small upload is failing because we are doing
> >>>> the same thing in the integration test and it works. I will spend some
> >>>> more time on this later tomorrow. Adrian any ideas? Have you seen
> >>>> something similar?
> >>>>
> >>>> -- Andrei Savu / andreisavu.ro
> >>>>
> >>>> On Fri, Jul 29, 2011 at 12:06 AM, Karel Vervaeke <
> karel@outerthought.org> wrote:
> >>>>> It is pretty consistent.
> >>>>>
> >>>>> Today there's something new on the menu...
> >>>>> http://pastebin.com/MSKDn6rt
> >>>>> (strange, because that's using the same file I was yesterday, so
> >>>>> nothing really changed except today's date)
> >>>>>
> >>>>> The number of bytes already written (mentioned in the stack trace)
is
> >>>>> always different.
> >>>>>
> >>>>> When I replace the file with something very small (184 bytes, to
be
> exact),
> >>>>> I get the original error again.
> >>>>>
> >>>>> Regards,
> >>>>> Karel
> >>>>>
> >>>>> On Thu, Jul 28, 2011 at 7:17 PM, Andrei Savu <savu.andrei@gmail.com>
> wrote:
> >>>>>> I've wrote that code but this is the first time I see this error.
> >>>>>>
> >>>>>> I tend to believe it's somehow related to jclouds. Does it happen
> all
> >>>>>> the time? How about with different files?
> >>>>>>
> >>>>>> -- Andrei Savu / andreisavu.ro
> >>>>>>
> >>>>>> On Thu, Jul 28, 2011 at 8:59 AM, Karel Vervaeke <
> karel@outerthought.org> wrote:
> >>>>>>> I'm trying out the local file upload from WHIRR-220, but
I'm
> running
> >>>>>>> in to a problem:
> >>>>>>>
> >>>>>>> While calling prepareRemoteFileUrl(...) (during the install
phase),
> I
> >>>>>>> get this as the last bit of output from whirr on the console:
> >>>>>>> http://pastebin.com/LZ92AUhQ
> >>>>>>>
> >>>>>>> After that there's no more output on the console for several
> minutes
> >>>>>>> (I just killed the process after that)
> >>>>>>>
> >>>>>>> I have tried the BlobCacheTest, which seems to work fine
for my AWS
> settings.
> >>>>>>> Before I start digging into jcloud's code, does anyone have
any
> >>>>>>> thought as to what might cause this?
> >>>>>>>
> >>>>>>> --
> >>>>>>> Karel Vervaeke
> >>>>>>> http://outerthought.org/
> >>>>>>> Open Source Content Applications
> >>>>>>> Makers of Kauri, Daisy CMS and Lily
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Karel Vervaeke
> >>>>> http://outerthought.org/
> >>>>> Open Source Content Applications
> >>>>> Makers of Kauri, Daisy CMS and Lily
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Karel Vervaeke
> >>> http://outerthought.org/
> >>> Open Source Content Applications
> >>> Makers of Kauri, Daisy CMS and Lily
> >>>
> >>
> >>
> >>
> >> --
> >> Karel Vervaeke
> >> http://outerthought.org/
> >> Open Source Content Applications
> >> Makers of Kauri, Daisy CMS and Lily
> >>
> >
> >
> >
> > --
> > Karel Vervaeke
> > http://outerthought.org/
> > Open Source Content Applications
> > Makers of Kauri, Daisy CMS and Lily
> >
>
>
>
> --
> Karel Vervaeke
> http://outerthought.org/
> Open Source Content Applications
> Makers of Kauri, Daisy CMS and Lily
>

Mime
View raw message