[PATCH 2 of 3] [RFC] color: allow color and pager to work on Windows with 'color.mode=auto'

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_harbison at yahoo.com>
# Date 1489985292 14400
#      Mon Mar 20 00:48:12 2017 -0400
# Node ID 6e72bb689e57c985b138c580acb4cad8053dc180
# Parent  a32b379ffe151b8b094dcd67ef5b6bd551203dbe
[RFC] color: allow color and pager to work on Windows with 'color.mode=auto'

It took me awhile to figure out that '--config color.mode=ansi' would allow the
MSYS pager to print color, even though the default config colors fine if the
pager is disabled.  This is a hacky attempt to improve 'auto'.  The only thing
that I can't figure out is why 'pager=no' causes the output to print a green
background.  It seems that the lack of the on-the-fly update causes this.
Hopefully someone with more knowledge of color and pager knows, and/or can point
out a better way.

Previously, 'color.mode=auto' would color by setting console attributes, and
then print the text.  This doesn't help the pager at all.  This will print with
the ANSI color codes if a pager has been spawned, falling back to the previous
method otherwise.

Each of the built in `more` (more.com), python pager [1], and `less` from
GnuWin32 print the raw ANSI color codes to the screen instead of colorizing.
(The latter basically prints a blank screen if given '-R'.)  The `less` that
comes with MSYS on the other hand, will print color with or without '-R'.  I
expect that `more` and MSYS will be the most popular pagers, so this gives a
better experience for them out of the box.  If one of the more obscure pagers
are used, setting color.pagermode will disable color for paged output.

(As an aside, the fact that 'color.pagermode' is only consulted if the pager has
been spawned [2] gave me the impression that color.setup() could be called
multiple times, given that color is setup so early.  But it actually is
triggered when --pager=yes is given.  The fact that this setting only works if
paging is forced with --pager seems... surprising.)

The _effects map is no longer updated directly because some values were getting
overwritten with Win32 values on the first call to color.setup(), which
prevented the color codes from being output when changing the ANSI when the
pager is spawned.

[1] https://pypi.python.org/pypi/pager
[2] https://www.mercurial-scm.org/repo/hg/file/da7d19324b1e/mercurial/color.py#l205

diff --git a/mercurial/color.py b/mercurial/color.py
--- a/mercurial/color.py
+++ b/mercurial/color.py
@@ -72,6 +72,8 @@
     'white_background': 47,
     }
 
+_activeeffects = _effects.copy()
+
 _defaultstyles = {
     'grep.match': 'red bold',
     'grep.linenumber': 'green',
@@ -231,12 +233,14 @@
         if mode == realmode and ui.formatted():
             ui.warn(_('warning: failed to set color mode to %s\n') % mode)
 
+    _activeeffects = _effects.copy()
+
     if realmode == 'win32':
         ui._terminfoparams.clear()
         if not w32effects:
             modewarn()
             return None
-        _effects.update(w32effects)
+        _activeeffects.update(w32effects)
     elif realmode == 'ansi':
         ui._terminfoparams.clear()
     elif realmode == 'terminfo':
@@ -271,7 +275,7 @@
 
 def valideffect(ui, effect):
     'Determine if the effect is valid or not.'
-    return ((not ui._terminfoparams and effect in _effects)
+    return ((not ui._terminfoparams and effect in _activeeffects)
              or (effect in ui._terminfoparams
                  or effect[:-11] in ui._terminfoparams))
 
@@ -305,9 +309,9 @@
                         for effect in ['none'] + effects.split())
         stop = _effect_str(ui, 'none')
     else:
-        start = [str(_effects[e]) for e in ['none'] + effects.split()]
+        start = [str(_activeeffects[e]) for e in ['none'] + effects.split()]
         start = '\033[' + ';'.join(start) + 'm'
-        stop = '\033[' + str(_effects['none']) + 'm'
+        stop = '\033[' + str(_activeeffects['none']) + 'm'
     return ''.join([start, text, stop])
 
 def colorlabel(ui, msg, label):
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -854,6 +854,12 @@
         # auto-detection of things being formatted.
         self.setconfig('ui', 'formatted', self.formatted(), 'pager')
         self.setconfig('ui', 'interactive', False, 'pager')
+
+        if self._colormode == 'win32' and not pagercmd.startswith('more.com'):
+            if not self.hasconfig('color', 'pagermode'):
+                self.setconfig('color', 'pagermode', 'ansi', source='pager')
+                color.setup(self)
+
         if util.safehasattr(signal, "SIGPIPE"):
             signal.signal(signal.SIGPIPE, _catchterm)
         self._runpager(pagercmd)