incubator-droids-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "" <>
Subject Re: About Link and POST requests
Date Wed, 27 Apr 2011 13:32:30 GMT
Hello Fuad,

I can agree that robots should not perform POST requests, but Droids
is a library for building robots, it is not a robot itself.
My suggestion to extends the request types was to widen the scope
where Droids could be useful.

I would find that an unfortunate decision, but that's just my point of view.


Giulio Cesare

On Wed, Apr 27, 2011 at 1:55 PM, Fuad Efendi <> wrote:
> Hi Giulio, "other request types" - HEAD only, and only if Internet Server supports that
(I don't think more than 1% support, and I don't think it will be performance improvement
in an era of sitemaps)... there is no "Link" request types, and "POST" shouldn't be done by
robots - otherwise which URL will you show in a search results page? You can't show form submission
via POST as a search results
> -Fuad
> -----Original Message-----
> From: []
> Sent: April-27-11 7:33 AM
> To:
> Subject: About Link and POST requests
> Hello,
> using Droid library I came to the realization that the current API are not suitable for
handling anything other than GET requests.
> This is due mainly to two choices made in the code:
> - using the Link interface for collecting and processing pages to process (all based
on simple URI values, both for 'getURI' and 'getTo');
> - content being loaded by CrawlingWorker, passing just the URI to the Protocol.
> Is there anyone interested in thinking how to change the API to support also other request
> Regards,
> Giulio Cesare

View raw message