incubator-droids-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From florent andré <florent.andre-...@4sengines.com>
Subject Re: About Link and POST requests
Date Wed, 27 Apr 2011 21:12:17 GMT
yep,

Actually some tools help to fill common / well identify forms fields 
(like email, phone, etc...)...

So it's totally reasonable to think about a robot with some "how to fill 
form" rules.
And could be valuable in some cases.

++

On 04/27/2011 03:32 PM, giulio.cesare@gmail.com wrote:
> Hello Fuad,
>
> I can agree that robots should not perform POST requests, but Droids
> is a library for building robots, it is not a robot itself.
> My suggestion to extends the request types was to widen the scope
> where Droids could be useful.
>
> I would find that an unfortunate decision, but that's just my point of view.
>
> Thanks,
>
> Giulio Cesare
>
>
> On Wed, Apr 27, 2011 at 1:55 PM, Fuad Efendi<fuad@efendi.ca>  wrote:
>> Hi Giulio, "other request types" - HEAD only, and only if Internet Server supports
that (I don't think more than 1% support, and I don't think it will be performance improvement
in an era of sitemaps)... there is no "Link" request types, and "POST" shouldn't be done by
robots - otherwise which URL will you show in a search results page? You can't show form submission
via POST as a search results
>> -Fuad
>>
>> -----Original Message-----
>> From: giulio.cesare@gmail.com [mailto:giulio.cesare@gmail.com]
>> Sent: April-27-11 7:33 AM
>> To: droids-dev@incubator.apache.org
>> Subject: About Link and POST requests
>>
>> Hello,
>>
>> using Droid library I came to the realization that the current API are not suitable
for handling anything other than GET requests.
>>
>> This is due mainly to two choices made in the code:
>> - using the Link interface for collecting and processing pages to process (all based
on simple URI values, both for 'getURI' and 'getTo');
>> - content being loaded by CrawlingWorker, passing just the URI to the Protocol.
>>
>> Is there anyone interested in thinking how to change the API to support also other
request types?
>>
>> Regards,
>>
>> Giulio Cesare
>>
>>

Mime
View raw message