incubator-droids-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "giulio.cesare@gmail.com" <giulio.ces...@gmail.com>
Subject Re: About Link and POST requests
Date Wed, 27 Apr 2011 13:32:30 GMT
Hello Fuad,

I can agree that robots should not perform POST requests, but Droids
is a library for building robots, it is not a robot itself.
My suggestion to extends the request types was to widen the scope
where Droids could be useful.

I would find that an unfortunate decision, but that's just my point of view.

Thanks,

Giulio Cesare


On Wed, Apr 27, 2011 at 1:55 PM, Fuad Efendi <fuad@efendi.ca> wrote:
> Hi Giulio, "other request types" - HEAD only, and only if Internet Server supports that
(I don't think more than 1% support, and I don't think it will be performance improvement
in an era of sitemaps)... there is no "Link" request types, and "POST" shouldn't be done by
robots - otherwise which URL will you show in a search results page? You can't show form submission
via POST as a search results
> -Fuad
>
> -----Original Message-----
> From: giulio.cesare@gmail.com [mailto:giulio.cesare@gmail.com]
> Sent: April-27-11 7:33 AM
> To: droids-dev@incubator.apache.org
> Subject: About Link and POST requests
>
> Hello,
>
> using Droid library I came to the realization that the current API are not suitable for
handling anything other than GET requests.
>
> This is due mainly to two choices made in the code:
> - using the Link interface for collecting and processing pages to process (all based
on simple URI values, both for 'getURI' and 'getTo');
> - content being loaded by CrawlingWorker, passing just the URI to the Protocol.
>
> Is there anyone interested in thinking how to change the API to support also other request
types?
>
> Regards,
>
> Giulio Cesare
>
>

Mime
View raw message