[PATCH 12 of 14 V3] streamclone: add support for cloning non append-only file

[Yuya Nishihara]

On Thu, 25 Jan 2018 09:35:38 +0100, Boris Feld wrote:
> On Sat, 2018-01-20 at 19:53 +0900, Yuya Nishihara wrote:
> > On Sat, 20 Jan 2018 19:41:23 +0900, Yuya Nishihara wrote:
> > > On Sat, 20 Jan 2018 00:47:17 +0100, Boris Feld wrote:
> > > > # HG changeset patch
> > > > # User Boris Feld <boris.feld at octobus.net>
> > > > # Date 1516233002 -3600
> > > > #      Thu Jan 18 00:50:02 2018 +0100
> > > > # Node ID 789f14aef1ca0b39e61a47d8989c18cdc6015b53
> > > > # Parent  d8a918033dcfd3dcbac8635616cc1b6c12078b18
> > > > # EXP-Topic b2-stream
> > > > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > > > #              hg pull https://bitbucket.org/octobus/mercurial-de
> > > > vel/ -r 789f14aef1ca
> > > > streamclone: add support for cloning non append-only file
> > > > +# type of file to stream
> > > > +_fileappend = 0 # append only file
> > > > +_filefull = 1   # full snapshot file
> > > > +
> > > > +# This is it's own function so extensions can override it.
> > > > +def _walkstreamfullstorefiles(repo):
> > > > +    """list snapshot file from the store"""
> > > > +    fnames = []
> > > > +    if not repo.publishing():
> > > > +        fnames.append('phaseroots')
> > > > +    return fnames
> > > > +
> > > > +def _filterfull(entry, copy, vfs):
> > > > +    """actually copy the snapshot files"""
> > > > +    name, ftype, data = entry
> > > > +    if ftype != _filefull:
> > > > +        return entry
> > > > +    return (name, ftype, copy(vfs.join(name)))
> > > > +
> > > > + at contextlib.contextmanager
> > > > +def maketempcopies():
> > > > +    """return a function to temporary copy file"""
> > > > +    files = []
> > > > +    try:
> > > > +        def copy(src):
> > > > +            fd, dst = tempfile.mkstemp()
> > > > +            os.close(fd)
> > > > +            files.append(dst)
> > > > +            util.copyfiles(src, dst, hardlink=True)
> > > 
> > > Better to create a temp file near the repository if we want to take
> > > advantage
> > > of hardlinks. /tmp may be a separate filesystem.
> > 
> > Ah, but hardlink will fail anyway since destination file exists.
> 
> Hardlinks are an option since we create the temporary file just for the
> streaming. We might use a temporary directory inside the 
> repository for this purpose.
> 
> It would be an improvement, but it adds some complexity and requires
> having a clear distinction between files we can hardlink and those we
> can't.
> 
> We think it would be better if we can ship the current version in 4.5,
> so we have the next cycle to improve it properly.

So maybe we should just turning off the hardlink flag for clarity?
It doesn't work if "dst" exists, and the "dst" must be created by mkstemp().