jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: JMeter 4.0RC... is still at http://home.apache.org/~milamber/jmeter-...
Date Thu, 03 Oct 2019 11:55:42 GMT

Oups... so I removed the robots.txt files and added a .htaccess file for 
redirect to https://jmeter.apache.org/



On 03/10/2019 09:45, Vladimir Sitnikov wrote:
> Milamber>I just upload a robots.txt file with disallow all into each RC
> into my
> Milamber>public html folder on home.apache.org.
>
> Can you please clarify?
>
> https://www.robotstxt.org/robotstxt.html says the file should be placed at
> the root directory.
>
> Then Google's documentation says:
>
> https://support.google.com/webmasters/answer/6062608?hl=en
> Google>A robotted page can still be indexed if linked to from from other
> sites
> Google>While Google won't crawl or index the content blocked by robots.txt,
> we might still find and index a disallowed URL if it is linked from other
> places on the web
>
> Which means "Google would discover the link to the preview from a mailing
> list archive", so the only feasible option is to remove the page completely
> (or move it to a new place that is never mentioned in the mailing lists"
>
> Vladimir
>


Mime
View raw message