robots.txt for www.mercurial-scm.org

Augie Fackler raf at durin42.com
Thu Sep 21 10:40:25 EDT 2017


> On Sep 19, 2017, at 12:28, Michael Bolin <bolinfest at gmail.com> wrote:
> 
> It appears that https://www.mercurial-scm.org/robots.txt disallows indexing of /repo. This makes it impossible to Google for Mercurial commits.
> 
> Is this an oversight or is it intentional?

(+ some folks)

I'm not sure offhand. The robots.txt is part of the website, but it hasn't been touched since 2014:

https://bitbucket.org/seanfarley/hgwebsite/history-node/ddff2dfaa572/static/robots.txt?at=default

David, do you remember why we ban crawling that area? I assume this is inherited from mpm, but maybe you remember something I never knew.

> 
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel



More information about the Mercurial-devel mailing list