robots.txt for www.mercurial-scm.org
Augie Fackler
raf at durin42.com
Thu Sep 21 10:40:25 EDT 2017
> On Sep 19, 2017, at 12:28, Michael Bolin <bolinfest at gmail.com> wrote:
>
> It appears that https://www.mercurial-scm.org/robots.txt disallows indexing of /repo. This makes it impossible to Google for Mercurial commits.
>
> Is this an oversight or is it intentional?
(+ some folks)
I'm not sure offhand. The robots.txt is part of the website, but it hasn't been touched since 2014:
https://bitbucket.org/seanfarley/hgwebsite/history-node/ddff2dfaa572/static/robots.txt?at=default
David, do you remember why we ban crawling that area? I assume this is inherited from mpm, but maybe you remember something I never knew.
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
More information about the Mercurial-devel
mailing list