<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 6, 2017 at 8:14 PM, Siddharth Agarwal <span dir="ltr"><<a href="mailto:sid@less-broken.com" target="_blank">sid@less-broken.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 3/6/17 09:50, Olivier Trempe wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
# HG changeset patch<br>
# User Olivier Trempe <<a href="mailto:oliviertrempe@gmail.com" target="_blank">oliviertrempe@gmail.com</a>><br>
# Date 1488810111 18000<br>
# Mon Mar 06 09:21:51 2017 -0500<br>
# Branch stable<br>
# Node ID c9d3f8d1a57346228f5c3bb749acdf<wbr>f90d37e194<br>
# Parent 6b00c3ecd15b26587de8cca6fab811<wbr>069cba3b2f<br>
fsmonitor: match watchman and local encoding<br>
<br>
watchman's paths encoding is os dependant. For example, on Windows, it's<br>
always utf-8. This causes paths comparison mismatch when paths contain non ascii<br>
characters.<br>
</blockquote>
<br>
I really doubt this is correct unixes, where Watchman returns bytes as they are on disk, which matches exactly with what Mercurial wants.<br>
<br>
(On Windows Watchman indeed always returns UTF-8.)<br>
<br></blockquote><div>This is what I meant by "os dependent". You get a different behavior on a different os. I can rewrite it to be more Windows specific. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
diff -r 6b00c3ecd15b -r c9d3f8d1a573 hgext/fsmonitor/__init__.py<br>
--- a/hgext/fsmonitor/__init__.py Thu Mar 02 20:19:45 2017 -0500<br>
+++ b/hgext/fsmonitor/__init__.py Mon Mar 06 09:21:51 2017 -0500<br>
@@ -99,6 +99,7 @@<br>
from mercurial import (<br>
context,<br>
encoding,<br>
+ error,<br>
extensions,<br>
localrepo,<br>
merge,<br>
@@ -110,6 +111,7 @@<br>
from mercurial import match as matchmod<br>
from . import (<br>
+ pywatchman,<br>
state,<br>
watchmanclient,<br>
)<br>
@@ -159,6 +161,20 @@<br>
sha1.update('\0')<br>
return sha1.hexdigest()<br>
+def _watchmanencodingtolocal(path)<wbr>:<br>
+ """Fix path to match watchman and local encoding<br>
+<br>
+ watchman's paths encoding is os dependant. For example, on Windows, it's<br>
</blockquote>
<br>
"dependent"<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+ always utf-8. This converts watchman encoded paths to local encoding to<br>
+ avoid paths comparison mismatch.<br>
+ """<br>
+ try:<br>
+ decoded = pywatchman.encoding.decode_loc<wbr>al(path)<br>
+ except UnicodeDecodeError as e:<br>
+ raise error.Abort(e, hint='watchman encoding error')<br>
</blockquote>
<br>
Could you elaborate a bit on when the exception can happen?<br></blockquote><div><br></div><div>This is a defensive implementation. I did not encounter this exception myself. However, even though watchman and pywatchman are maintained together, there is no guarantee that the user will run the watchman build matching the pywatchman version distributed with mercurial. Given that getting the watchman encoding relies on pywatchman's encoding module, you could get an error if something changes on either side. I just didn't want to let a possible unhandled exception propagate up to the user.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+<br>
+ return decoded.encode(encoding.encodi<wbr>ng, 'replace')<br>
+<br>
def overridewalk(orig, self, match, subrepos, unknown, ignored, full=True):<br>
'''Replacement for dirstate.walk, hooking into Watchman.<br>
@@ -302,7 +318,7 @@<br>
# Watchman tracks files. We use this property to reconcile deletes<br>
# for name case changes.<br>
for entry in result['files']:<br>
- fname = entry['name']<br>
+ fname = _watchmanencodingtolocal(entry<wbr>['name'])<br>
</blockquote>
<br>
Adding a non-trivial function call here is likely going to regress performance. Have you measured the impact?</blockquote><div><br></div><div>100000 calls to the function result in a ~0.3 seconds impact.</div><div>I can optimize it by triggering the conversion only if the encoding returned by pywatchman is different than the local encoding. If they are the same, the impact is close to null. If not, the impact is ~0.24 seconds.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
- Siddharth<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
if switch_slashes:<br>
fname = fname.replace('\\', '/')<br>
if normalize:<br>
______________________________<wbr>_________________<br>
Mercurial-devel mailing list<br>
<a href="mailto:Mercurial-devel@mercurial-scm.org" target="_blank">Mercurial-devel@mercurial-scm.<wbr>org</a><br>
<a href="https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel" rel="noreferrer" target="_blank">https://www.mercurial-scm.org/<wbr>mailman/listinfo/mercurial-dev<wbr>el</a><br>
</blockquote>
<br>
<br>
</blockquote></div><br></div><div class="gmail_extra">This is a very nasty bug on windows. In common cases, you don't get modified files when calling status. No so bad. However, I lost modifications while doing history editing operations. Files containing non-ascii characters in their path were silently discarded from a rebased changeset.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Thanks for your comments. I will submit a new patch when I get feedback from you.</div></div>