Bug 5807 - Convert throws "ValueError: I/O operation on closed file"
Summary: Convert throws "ValueError: I/O operation on closed file"
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: convert (show other bugs)
Version: 4.2
Hardware: PC Linux
: wish bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-01 05:02 UTC by Sascha Nemecek
Modified: 2018-04-03 00:00 UTC (History)
3 users (show)

See Also:
Python Version: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sascha Nemecek 2018-03-01 05:02 UTC
= Background =

Due to a recent system upgrade on one of our machines (openSuSE 42.2 -> 42.3), mercurial was updated from 3.8.3 to 4.2.3. After that, our script to fetch, update and convert a remote SVN repo into our local HG mirror throws the following error:

> ** Unknown exception encountered with possibly-broken third-party extension prompt
> ** which supports versions unknown of Mercurial.
> ** Please disable prompt and try your action again.
> ** If that fixes the bug please report it to the extension author.
> ** Python 2.7.13 (default, Jan 03 2017, 17:41:54) [GCC]
> ** Mercurial Distributed SCM (version 4.2.3)
> ** Extensions loaded: convert, extdiff, churn, prompt
> Traceback (most recent call last):
>   File "/usr/bin/hg", line 45, in <module>
>     mercurial.dispatch.run()
>   File "/usr/lib64/python2.7/site-packages/mercurial/dispatch.py", line 83, in run
>     status = (dispatch(req) or 0) & 255
>   File "/usr/lib64/python2.7/site-packages/mercurial/dispatch.py", line 178, in dispatch
>     req.ui.flush()
>   File "/usr/lib64/python2.7/site-packages/mercurial/ui.py", line 823, in flush
>     self.fout.flush()
> ValueError: I/O operation on closed file


= Findings =

1. Disabling the prompt extension did not improve the situation (same error persists).

2. We tested with the following versions:
   - 3.8.3-2.11.1: worked prior upgrade
   - 4.2-1.2:      throws error shown above
   - 4.2.3-7.1:    throws error shown above
   - 4.5-202.2:    throws error shown above

3. Error suggests that the targeted file or buffer is closed prematurely. Even when inserting a python debug before the self.fout.flush() statement, the error is triggered.

4. After investigating the code, I think the problem stems from changeset 30261:6bed17ba00a1 (see https://www.mercurial-scm.org/repo/hg/rev/6bed17ba00a1). It replaced sys.stdin / sys.stdout with ui.fin / ui.fout. But it did not take into consideration, that the buffer automatically is closed in "get_log_child()".


= Proposed change =

In "get_log_child()", I would simply replace the fp.close() with fp.flush(). This will ensure that the buffer is still available.

I'll submit a patch to mercurial-devel@mercurial-scm.org soonish.
Comment 1 Sascha Nemecek 2018-03-02 05:16 UTC
Tested three more mercurial versions:
- 3.9.1: no error
- 4.0:   no error
- 4.4.1: throws error shown above
Comment 2 HG Bot 2018-03-02 19:20 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/d697e39f61a6
Sascha Nemecek <nemecek@wienfluss.net>
convert: avoid closing ui.fout in subversion code (issue5807)

Don't close 'fp' (= 'ui.fout') stream to prevent 'ValueError: I/O
operation on closed file' (Bug #5807).

Regression of changeset 30261:6bed17ba00a1
(https://www.mercurial-scm.org/repo/hg/rev/6bed17ba00a1)

(please test the fix)
Comment 3 Yuya Nishihara 2018-03-26 09:44 UTC
(no idea why this was left as UNCONFIRMED.)
Comment 4 Bugzilla 2018-04-03 00:00 UTC
Bug was set to TESTING for 7 days, resolving