Output buffering on Windows 10

Sune Foldager sune.foldager at me.com
Thu Jun 14 13:24:12 UTC 2018


Greetings

I’d like to discuss a problem which results in console output not working correctly on Windows 10 and not having done  so since late may 2017! The actual cause is due to a changeset by Simon Farnsworth in the beginning of february last year (changeset 3a4c0905f357):

util: always force line buffered stdout when stdout is a tty (BC)

…which essentially causes stdout to be reopened in line buffered mode as long as stdout is a tty at program start. This was previously only done when the pager was activated (in pager.py). Apparently this was done to make some later changes (by Simon) easier. The reopen happens in util.py:

+if isatty(stdout):
+    stdout = os.fdopen(stdout.fileno(), 'wb', 1)

On Linux, I guess this is a no-op since the default buffering mode for ttys is line buffering. Unfortunately this doesn’t work on Windows. The default on Windows is unbuffered and furthermore Windows doesn’t even support line buffering mode, treating it as fully buffered.

Now, normally this wouldn’t affect Windows since the “win32” color mode was used there, which involves breaking the text down and calling flush a lot (at least once after each ui.write). But now comes Matt Harbison’s change (changeset 98c2b44bdf9a) which enables ANSI mode on Windows, causing it to use the same code paths as Linux. This activates the bug.

With the bug active, all output is fully buffered on Windows which can readily be seen by doing something “slow” like hg in, hg clone or similar. The end result is very unsatisfactory. As a workaround (at my workplace), we have set our color.mode back to win32 (instead of auto) and specified the color.pagermode manually to ansi. But we’d obviously like to use ANSI mode, now that it’s supported by Windows.

I’d love to fix this bug, but I don’t know how, since the most obvious solution would be to exactly back out of Simon’s changeset, only enabling buffered mode (full on Windows, line on Linux) when the pager is about to run. But Simon clearly didn’t make that patch for nothing so… I’m not sure what the best course of action is.

I am also a bit puzzled that no one has noticed this bug, e.g. Mark when he enabled this behaviour, but of course it doesn’t happen for “fast” commands like hg debugcolor or similar.

Thanks :)

— 
Sune



More information about the Mercurial-devel mailing list