disappearing repo history on samba server

[Adrian Buehlmann]

On 2010-12-07 23:29, Joel B. Mohler wrote:
> On 12/7/2010 5:20 PM, Adrian Buehlmann wrote:
>> On 2010-12-07 21:42, Adrian Buehlmann wrote:
>>> On 2010-12-07 20:38, Joel B. Mohler wrote:
>>>> Hi,  My windows machines have been having a rocky relationship with
>>>> their samba server.  The latest issue is that a push to a certain
>>>> repository will wipe out revision history.  I've narrowed this down to a
>>>> very small slice of mercurial code which has nothing at all to do with
>>>> the changeset index, but the symptom with-in mercurial is that
>>>> .hg\store\00changelog.i is renamed to .hg\store\00changelog.i.hgtmp and
>>>> then my repository history is effectively wiped out (although the exact
>>>> details of what goes on with 00changelog.i.hgtmp appear to vary somewhat
>>>> by windows client machine).
>>>>
>>>> The fatal bug is reproduced by the python script at
>>>>       http://kiwistrawberry.us/opener.py
>>>> This script assumes that you have mercurial and python win32 extensions
>>>> installed on a windows machine.  Without python win32 extensions, the
>>>> windows link code goes into graceful (?) degradation mode and so the
>>>> faulty code is not run.  The bug reproduces with two different windows
>>>> machines (vista and server 2008) and two different linux servers (ubuntu
>>>> and gentoo) so I'm quite confident it's not just a server configuration
>>>> fluke.
>>>>
>>>> However, there's a weird gotcha which I don't understand yet.  It is
>>>> that I can only reproduce this on a large repository of about 32000
>>>> revisions which I can't make public.  Attempts on a smaller repository
>>>> have all worked (i.e. reproducing the bug failed).  From my opener.py
>>>> script, I come to the conclusion that for some reason 00changelog.i is
>>>> held open longer for a larger repository, but I was unable to determine
>>>> why that might be.
>>>>
>>>> All relevant mercurial installs are at least 1.7.1 and I don't see any
>>>> evidence that variations of revision beyond the arrival of the
>>>> checknlink function make any difference.
>>>>
>>>> So, two questions:
>>>> 1)  Am I correct in believing that opener.py illustrates potential for a
>>>> data-losing bug?
>>>> 2)  Is more information needed about my repository or can a fix for
>>>> opener.py be found with-out that?
>>>>
>>> Thanks for your http://kiwistrawberry.us/opener.py script, I'll take a
>>> closer look.
>> Or maybe not. I might not have time or motivation to look at this. If
>> anyone else has any ambitions or ideas here, feel free to jump in here.
> 
> I think I will pursue this some more myself.  I reproduced what appears 
> to me to be at least one part of the puzzle with-out mercurial code at 
> all.  The script in this link is merely python and pywin32: 
> http://kiwistrawberry.us/pywin32_link_issues.py

Very good. After all you are driven by your own immediate need, which is
by far the best motivation :). (I was just mainly concerned about
mercurial getting a bad reputation, since I would never ever put my own
repos on a Windows share :p).

I guess the few remaining Mercurial developers who care about Windows
don't care that much about repos on Windows shares (although the unlink
"ghost file" problem doesn't even require a Windows share to happen,
yikes...).

If I might make a suggestion: Matt usually prefers inline info and I
think pasting your short scripts here would indeed be helpful for others
to look at. And describe in prose what you found at each step. You can
also use the wiki.

Here is Joel's script (looks very interesting, combination of hardlink +
windows share + os.unlink !):

  import os
  import win32file

  dir,file=r"\\tcpkal\downloads","test_links.txt"
  f=os.path.join(dir,file)
  open(f,"w")

  win32file.CreateHardLink(f+".link",f)
  fd=open(f)

  raw_input("Try to delete the .link file now...")

  os.unlink(f+".link")
  fd.close()