No subject


Thu Apr 29 17:22:33 UTC 2010


the pull protocol on the Remote server B, which is the one that has the 64
bit Windows 2003 SP2 and which is also virtualized.

I repeated most of the tests several times and the results were quite
consistent. In some cases the worst case scenario time (remote machine B to
local, no pull) has been worse than 3 minutes (up to 5 minutes). On the
other hand the good scenarios have never been slow. I suspect that the worst
case scenario timing may depend on the load of the remote machine.

I also did a test in which I stopped the Web Client service, but this did
not seem to make a difference (i.e. the worst case scenario was still slow).

Regarding CIFS or SMB I am not sure which is being used. I don't think that
you can chose one or the other when you create a network share on Windows.

Finally, I performed a folder diff between two clones made from the Remote
server A (the one showing the problem). I had used the pull protocol to
create one of the clones, while for the other I did not use it. I was
surprised that there were some differences on the .hg folder:
- The "dirstate" files were different
- The clone for which I used "pull" had some extra files: undo.branch,
undor.dirstate and store/undo.

When I compared two clones that both had used the pull protocol the only
difference was on the "dirstate" file. This is probably normal, but I wanted
to point it out just in case it was useful to narrow down the problem even
further.

I'll try to install WireShark soon. When I do so I will try to capture the
problem. Is it ok to send the trace as an attachment to the list? It may be
a big attachment file...

Cheers,

Didly


On Fri, May 28, 2010 at 1:10 AM, Matt Mackall <mpm at selenic.com> wrote:

> On Thu, 2010-05-27 at 16:51 +0200, Didly Bom wrote:
> > On Thu, May 27, 2010 at 4:24 PM, Matt Mackall <mpm at selenic.com> wrote:
> >
> >
> >         When you say SMB, do you actually mean the completely
> >         different CIFS
> >         protocol?
> >
> >
> > Matt, I actually meant a regular windows network share. If I recall
> > correctly that used to be called a samba or smb share on the linux
> > world. My understanding was that cifs is just an evolution of smb.
>
> For the purposes of this discussion, it's very significant: CIFS
> supports reporting hardlinks and SMB doesn't. Using a mix of the two is
> known to be problematic because the SMB side will think there are no
> hardlinks to break and will thus corrupt your repository.
>
> I'm pretty sure thousands of people use Mercurial over Windows shares on
> a daily basis, so if there's something going on here, it's specific to
> your setup. Please tell us more. Specifically:
>
> - what operating system is on the server
> - what filesystem is in use on the server
> - precisely what network filesystem is being used (CIFS or SMB)
> - what virus scanner or similar is in use on clients and server
>
> Performance of a non-pull clone with hardlinks should be at least an
> order of magnitude faster than --pull. At worst, it should be no slower
> than the time to copy a .hg directory tree and then do a full checkout.
> You can time that manually and compare.
>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
>

--00163641780787bf490487a4b9ec
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Matt,<div><br></div><div>I did a few more tests which hopefully will narrow=
 down the problem. I performed the tests using 4 machines:</div><div><br></=
div><div>- Local machine: Microsoft Windows XP Professional (2002 version),=
 Service Pack 3, Spanish version. Dell Latitude D430, Intel Core2 (U7600) @=
 1.20 GHz, 2 GB RAM. =A0Eset NOD32 Antivirus (v4.0.314.0). Running WebClien=
t service.</div>
<div><br></div><div>- Remote machine A: Microsoft Windows Server 2003, Stan=
dard x64 Edition, Service Pack 2, Spanish version. Intell Xeon CPU (E5410) =
@ 2.33 GHz, 13.7 GB RAM.=A0Eset NOD32 Antivirus (v4.0.417.0). Not running W=
ebClient service. I believe that this server is virtualized (as all our win=
dows servers are).</div>
<div><br></div><div>- Remote machine B:=A0Microsoft Windows XP Professional=
 (2002 version), Service Pack 3, Spanish version. Intel Core2 (6400) @ 2.13=
 GHz, 3.50 GB RAM. =A0Eset NOD32 Antivirus (v4.0.314.0). Running WebClient =
service.</div>
<div><br></div><div>- Remote machine C: Red Hat Linux 5. 8 GB RAM. I believ=
e it has a quad core Xeon CPU but I am not sure. The Gnome System Monitor r=
eports 8 CPUs.</div><div><br></div><div>I used TortoiseHg running on the lo=
cal machine to perform the following operations. Note that in all cases Tor=
toiseHg was running in the machine called &quot;Local machine&quot;. The on=
ly difference between each of the tests is the source and target destinatio=
n addresses that I typed on the Clone tool of TortoiseHg and whether I enab=
led or not the &quot;Use pull protocol&quot; checkbox:</div>
<div><br></div><div>1.- When I make a clone of a 3 MB repository from Remot=
e machine A into the local machine with pull enabled it takes around 10 sec=
onds.</div><div>2.-=A0When I make a clone=A0from Remote machine A into the =
local machine but=A0I don&#39;t use the pull option, it takes between 2 and=
 a half and 3 minutes.</div>
<div>3.- When I make a clone from my local machine into the Remote machine =
A, using the pull option it takes around 30 seconds.</div><div>4.- When I m=
ake a clone from my local machine into the Remote machine A, without the pu=
ll option, it takes around 15 seconds.</div>
<div><br></div><div><div>5.- When I make a clone from Remote machine B into=
 the local machine with pull enabled it takes around 8 seconds.</div><div>6=
.-=A0When I make a clone=A0from Remote machine B into the local machine but=
=A0I don&#39;t use the pull option, it takes around 10 seconds.</div>
<div><br></div></div><div><div>7.- When I make a clone from Remote machine =
B into the local machine with pull enabled it takes around 8 seconds.</div>=
<div>8.-=A0When I make a clone=A0from Remote machine B into the local machi=
ne but=A0I don&#39;t use the pull option, it takes around 10 seconds.</div>
<div>9.- When I make a clone from my local machine into the Remote machine =
B, using the pull option it takes around 20 seconds.</div><div>10.- When I =
make a clone from my local machine into the Remote machine B, without the p=
ull option, it takes around 10 seconds.</div>
</div><div><br></div><div>11.- When I make a clone from Remote machine A in=
to the Remote machine B with pull enabled it takes around 30 seconds.</div>=
<div>12.- When I make a clone from Remote machine A into the Remote machine=
 B but I don&#39;t use the pull option it takes around 3 minutes.</div>
<div><br></div><div>From these tests it is clear that the problem only happ=
ens when I do not use the pull protocol on the Remote server B, which is th=
e one that has the 64 bit Windows 2003 SP2 and which is also virtualized.</=
div>
<div><br></div><div><div>I repeated most of the tests several times and the=
 results were quite consistent. In some cases the worst case scenario time =
(remote machine B to local, no pull) has been worse than 3 minutes (up to 5=
 minutes). On the other hand the good scenarios have never been slow. I sus=
pect that the worst case scenario timing may depend on the load of the remo=
te machine.</div>
<div><br></div><div>I also did a test in which I stopped the Web Client ser=
vice, but this did not seem to make a difference (i.e. the worst case scena=
rio was still slow).</div><div><br></div></div><div>Regarding CIFS or SMB I=
 am not sure which is being used. I don&#39;t think that you can chose one =
or the other when you create a network share on Windows.</div>
<div><br></div><div>Finally, I performed a folder diff between two clones m=
ade from the Remote server A (the one showing the problem). I had used the =
pull protocol to create one of the clones, while for the other I did not us=
e it. I was surprised that there were some differences on the .hg folder:</=
div>
<div>- The &quot;dirstate&quot; files were different</div><div>- The clone =
for which I used &quot;pull&quot; had some extra files: undo.branch, undor.=
dirstate and store/undo.</div><div><br></div><div>When I compared two clone=
s that both had used the pull protocol the only difference was on the &quot=
;dirstate&quot; file.=A0This is probably normal, but I wanted to point it o=
ut just in case it was useful to narrow down the problem even further.</div=
>
<div><br></div><div>I&#39;ll try to install WireShark soon. When I do so I =
will try to capture the problem. Is it ok to send the trace as an attachmen=
t to the list? It may be a big attachment file...</div><div><br></div><div>
Cheers,</div><div><br></div><div>Didly</div><div><br></div><div><br><div cl=
ass=3D"gmail_quote">On Fri, May 28, 2010 at 1:10 AM, Matt Mackall <span dir=
=3D"ltr">&lt;<a href=3D"mailto:mpm at selenic.com">mpm at selenic.com</a>&gt;</sp=
an> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;"><div class=3D"im">On Thu, 2010-05-27 at 16:=
51 +0200, Didly Bom wrote:<br>
&gt; On Thu, May 27, 2010 at 4:24 PM, Matt Mackall &lt;<a href=3D"mailto:mp=
m at selenic.com">mpm at selenic.com</a>&gt; wrote:<br>
&gt;<br>
&gt;<br>
&gt; =A0 =A0 =A0 =A0 When you say SMB, do you actually mean the completely<=
br>
&gt; =A0 =A0 =A0 =A0 different CIFS<br>
&gt; =A0 =A0 =A0 =A0 protocol?<br>
&gt;<br>
&gt;<br>
&gt; Matt, I actually meant a regular windows network share. If I recall<br=
>
&gt; correctly that used to be called a samba or smb share on the linux<br>
&gt; world. My understanding was that cifs is just an evolution of smb.<br>
<br>
</div>For the purposes of this discussion, it&#39;s very significant: CIFS<=
br>
supports reporting hardlinks and SMB doesn&#39;t. Using a mix of the two is=
<br>
known to be problematic because the SMB side will think there are no<br>
hardlinks to break and will thus corrupt your repository.<br>
<br>
I&#39;m pretty sure thousands of people use Mercurial over Windows shares o=
n<br>
a daily basis, so if there&#39;s something going on here, it&#39;s specific=
 to<br>
your setup. Please tell us more. Specifically:<br>
<br>
- what operating system is on the server<br>
- what filesystem is in use on the server<br>
- precisely what network filesystem is being used (CIFS or SMB)<br>
- what virus scanner or similar is in use on clients and server<br>
<br>
Performance of a non-pull clone with hardlinks should be at least an<br>
order of magnitude faster than --pull. At worst, it should be no slower<br>
than the time to copy a .hg directory tree and then do a full checkout.<br>
You can time that manually and compare.<br>
<div><div></div><div class=3D"h5"><br>
--<br>
Mathematics is the supreme nostalgia of our time.<br>
<br>
<br>
</div></div></blockquote></div><br></div>

--00163641780787bf490487a4b9ec--


More information about the Mercurial mailing list