[PATCH 12 of 14 V3] streamclone: add support for cloning non append-only file

[Yuya Nishihara]

On Sat, 20 Jan 2018 00:47:17 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.feld at octobus.net>
> # Date 1516233002 -3600
> #      Thu Jan 18 00:50:02 2018 +0100
> # Node ID 789f14aef1ca0b39e61a47d8989c18cdc6015b53
> # Parent  d8a918033dcfd3dcbac8635616cc1b6c12078b18
> # EXP-Topic b2-stream
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 789f14aef1ca
> streamclone: add support for cloning non append-only file

> +# type of file to stream
> +_fileappend = 0 # append only file
> +_filefull = 1   # full snapshot file
> +
> +# This is it's own function so extensions can override it.
> +def _walkstreamfullstorefiles(repo):
> +    """list snapshot file from the store"""
> +    fnames = []
> +    if not repo.publishing():
> +        fnames.append('phaseroots')
> +    return fnames
> +
> +def _filterfull(entry, copy, vfs):
> +    """actually copy the snapshot files"""
> +    name, ftype, data = entry
> +    if ftype != _filefull:
> +        return entry
> +    return (name, ftype, copy(vfs.join(name)))
> +
> + at contextlib.contextmanager
> +def maketempcopies():
> +    """return a function to temporary copy file"""
> +    files = []
> +    try:
> +        def copy(src):
> +            fd, dst = tempfile.mkstemp()
> +            os.close(fd)
> +            files.append(dst)
> +            util.copyfiles(src, dst, hardlink=True)

Better to create a temp file near the repository if we want to take advantage
of hardlinks. /tmp may be a separate filesystem.

> +    with maketempcopies() as copy:
> +        try:
> +            # copy is delayed until we are in the try
> +            entries = [_filterfull(e, copy, vfs) for e in entries]

I'm not sure if this is better or worse, but maybe we can hold open file
handles (instead of copying them) given "full" files are atomically replaced
on write.