D5122: remotefilelog: remove support for fetching packs
durin42 (Augie Fackler)
phabricator at mercurial-scm.org
Mon Nov 5 19:55:14 UTC 2018
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6d64e2abe8d3: remotefilelog: remove support for fetching packs (authored by durin42, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5122?vs=12178&id=12414
REVISION DETAIL
https://phab.mercurial-scm.org/D5122
AFFECTED FILES
hgext/remotefilelog/__init__.py
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/shallowrepo.py
tests/test-remotefilelog-prefetch.t
tests/test-remotefilelog-repack-fast.t
tests/test-remotefilelog-repack.t
CHANGE DETAILS
diff --git a/tests/test-remotefilelog-repack.t b/tests/test-remotefilelog-repack.t
--- a/tests/test-remotefilelog-repack.t
+++ b/tests/test-remotefilelog-repack.t
@@ -293,38 +293,32 @@
> [remotefilelog]
> data.generations=60
> 150
- > fetchpacks=True
> EOF
Single pack - repack does nothing
$ hg prefetch -r 0
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
+ [1]
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
+ [1]
$ hg repack --incremental
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
3 gen1 packs, 1 gen0 pack - packs 3 gen1 into 1
$ hg prefetch -r 1
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 2
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
- $ hg prefetch -r 3
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
+ $ hg prefetch -r 38
+ abort: unknown revision '38'!
+ [255]
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack
- -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack
- -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
- -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack
-r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
For the data packs, setting the limit for the repackmaxpacksize to be 64 such
@@ -335,58 +329,46 @@
$ hg repack --incremental --config remotefilelog.data.repackmaxpacksize=64 \
> --config remotefilelog.history.repackmaxpacksize=0
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack
- -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack
- -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack
+ -r--r--r-- 147 935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
- -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack
-r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
Setting limit for the repackmaxpacksize to be the size of the biggest pack file
which ensures that it is effectively ignored in the incremental repacking.
$ hg repack --incremental --config remotefilelog.data.repackmaxpacksize=65 \
> --config remotefilelog.history.repackmaxpacksize=336
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 147 935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
+ -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
+ -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
1 gen3 pack, 1 gen0 pack - does nothing
$ hg repack --incremental
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 147 935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
+ -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
+ -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
Pull should run background repack
$ cat >> .hg/hgrc <<EOF
> [remotefilelog]
> backgroundrepack=True
> EOF
$ clearcache
$ hg prefetch -r 0
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 1
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 2
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 3
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
- $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack
- -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack
- -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack
- $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
- -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack
- -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg pull
pulling from ssh://user@dummy/master
@@ -396,26 +378,24 @@
$ sleep 0.5
$ hg debugwaitonrepack >/dev/null 2>&1
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 301 09b8bf49256b3fc2175977ba97d6402e91a9a604.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
Test environment variable resolution
$ CACHEPATH=$TESTTMP/envcache hg prefetch --config 'remotefilelog.cachepath=$CACHEPATH'
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ find $TESTTMP/envcache | sort
$TESTTMP/envcache
$TESTTMP/envcache/master
- $TESTTMP/envcache/master/packs
- $TESTTMP/envcache/master/packs/54afbfda203716c1aa2636029ccc0df18165129e.dataidx
- $TESTTMP/envcache/master/packs/54afbfda203716c1aa2636029ccc0df18165129e.datapack
- $TESTTMP/envcache/master/packs/dcebd8e8d4d97ee88e40dd8f92d8678c10e1a3ad.histidx
- $TESTTMP/envcache/master/packs/dcebd8e8d4d97ee88e40dd8f92d8678c10e1a3ad.histpack
+ $TESTTMP/envcache/master/95
+ $TESTTMP/envcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a
+ $TESTTMP/envcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/577959738234a1eb241ed3ed4b22a575833f56e0
+ $TESTTMP/envcache/repos
Test local remotefilelog blob is correct when based on a pack
$ hg prefetch -r .
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ echo >> y
$ hg commit -m y2
$ hg debugremotefilelog .hg/store/data/95cb0bfd2977c761298d9624e4b4d4c72a39974a/b70860edba4f8242a1d52f2a94679dd23cb76808
@@ -434,7 +414,7 @@
Test limiting the max delta chain length
$ hg repack --config packs.maxchainlen=1
$ hg debugdatapack $TESTTMP/hgcache/master/packs/*.dataidx
- $TESTTMP/hgcache/master/packs/a2731c9a16403457b67337a620931797fce8c821:
+ $TESTTMP/hgcache/master/packs/a8378d888fe4325250720e51311a65d2509be742:
x:
Node Delta Base Delta Length Blob Size
1bb2e6237e03 000000000000 8 8
@@ -445,20 +425,20 @@
Total: 36 20 (80.0% bigger)
y:
Node Delta Base Delta Length Blob Size
- 577959738234 000000000000 8 8
+ 577959738234 000000000000 70 8
- Total: 8 8 (0.0% bigger)
+ Total: 70 8 (775.0% bigger)
Test huge pack cleanup using different values of packs.maxpacksize:
$ hg repack --incremental --debug
$ hg repack --incremental --debug --config packs.maxpacksize=512
- removing oversize packfile $TESTTMP/hgcache/master/packs/a2731c9a16403457b67337a620931797fce8c821.datapack (365 bytes)
- removing oversize packfile $TESTTMP/hgcache/master/packs/a2731c9a16403457b67337a620931797fce8c821.dataidx (1.21 KB)
+ removing oversize packfile $TESTTMP/hgcache/master/packs/a8378d888fe4325250720e51311a65d2509be742.datapack (426 bytes)
+ removing oversize packfile $TESTTMP/hgcache/master/packs/a8378d888fe4325250720e51311a65d2509be742.dataidx (1.21 KB)
Do a repack where the new pack reuses a delta from the old pack
$ clearcache
$ hg prefetch -r '2::3'
- 2 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over * (glob)
$ hg repack
$ hg debugdatapack $CACHEDIR/master/packs/*.datapack
$TESTTMP/hgcache/master/packs/abf210f6c3aa4dd0ecc7033633ad73591be16c95:
@@ -469,7 +449,7 @@
Total: 20 14 (42.9% bigger)
$ hg prefetch -r '0::1'
- 2 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over * (glob)
$ hg repack
$ hg debugdatapack $CACHEDIR/master/packs/*.datapack
$TESTTMP/hgcache/master/packs/09b8bf49256b3fc2175977ba97d6402e91a9a604:
diff --git a/tests/test-remotefilelog-repack-fast.t b/tests/test-remotefilelog-repack-fast.t
--- a/tests/test-remotefilelog-repack-fast.t
+++ b/tests/test-remotefilelog-repack-fast.t
@@ -285,78 +285,63 @@
> [remotefilelog]
> data.generations=60
> 150
- > fetchpacks=True
> EOF
Single pack - repack does nothing
$ hg prefetch -r 0
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
+ [1]
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
+ [1]
$ hg repack --incremental
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
3 gen1 packs, 1 gen0 pack - packs 3 gen1 into 1
$ hg prefetch -r 1
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 2
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 3
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack
- -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack
- -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
- -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack
-r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
$ hg repack --incremental
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
-r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
+ -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
1 gen3 pack, 1 gen0 pack - does nothing
$ hg repack --incremental
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
-r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
+ -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
Pull should run background repack
$ cat >> .hg/hgrc <<EOF
> [remotefilelog]
> backgroundrepack=True
> EOF
$ clearcache
$ hg prefetch -r 0
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 1
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 2
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg prefetch -r 3
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
- $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack
- -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack
- -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack
- $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
- -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack
- -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
- -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack
- -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ hg pull
pulling from ssh://user@dummy/master
@@ -366,26 +351,24 @@
$ sleep 0.5
$ hg debugwaitonrepack >/dev/null 2>&1
$ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack
- -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack
- -r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack
+ -r--r--r-- 301 09b8bf49256b3fc2175977ba97d6402e91a9a604.datapack
$ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack
-r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack
Test environment variable resolution
$ CACHEPATH=$TESTTMP/envcache hg prefetch --config 'remotefilelog.cachepath=$CACHEPATH'
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ find $TESTTMP/envcache | sort
$TESTTMP/envcache
$TESTTMP/envcache/master
- $TESTTMP/envcache/master/packs
- $TESTTMP/envcache/master/packs/54afbfda203716c1aa2636029ccc0df18165129e.dataidx
- $TESTTMP/envcache/master/packs/54afbfda203716c1aa2636029ccc0df18165129e.datapack
- $TESTTMP/envcache/master/packs/dcebd8e8d4d97ee88e40dd8f92d8678c10e1a3ad.histidx
- $TESTTMP/envcache/master/packs/dcebd8e8d4d97ee88e40dd8f92d8678c10e1a3ad.histpack
+ $TESTTMP/envcache/master/95
+ $TESTTMP/envcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a
+ $TESTTMP/envcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/577959738234a1eb241ed3ed4b22a575833f56e0
+ $TESTTMP/envcache/repos
Test local remotefilelog blob is correct when based on a pack
$ hg prefetch -r .
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ echo >> y
$ hg commit -m y2
$ hg debugremotefilelog .hg/store/data/95cb0bfd2977c761298d9624e4b4d4c72a39974a/b70860edba4f8242a1d52f2a94679dd23cb76808
diff --git a/tests/test-remotefilelog-prefetch.t b/tests/test-remotefilelog-prefetch.t
--- a/tests/test-remotefilelog-prefetch.t
+++ b/tests/test-remotefilelog-prefetch.t
@@ -201,66 +201,36 @@
$ clearcache
$ hg addremove -s 50 > /dev/null
3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over * (glob)
-
- $ cd ..
+ $ hg revert --all
+ forgetting x2
+ forgetting y2
+ forgetting z2
+ undeleting x
+ undeleting y
+ undeleting z
-# Prefetch packs
- $ hgcloneshallow ssh://user@dummy/master packprefetch
- streaming all changes
- 2 files to transfer, 528 bytes of data
- transferred 528 bytes in * seconds (*/sec) (glob)
- searching for changes
- no changes found
- updating to branch default
- 3 files updated, 0 files merged, 0 files removed, 0 files unresolved
- $ cd packprefetch
- $ cat >> .hg/hgrc <<EOF
- > [remotefilelog]
- > fetchpacks=True
- > backgroundrepack=True
- > EOF
- $ clearcache
- $ hg prefetch -r .
- 3 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
- $ find $TESTTMP/hgcache -type f | sort
- $TESTTMP/hgcache/master/packs/47d8f1b90a73af4ff8af19fcd10bdc027b6a881a.histidx
- $TESTTMP/hgcache/master/packs/47d8f1b90a73af4ff8af19fcd10bdc027b6a881a.histpack
- $TESTTMP/hgcache/master/packs/8c654541e4f20141a894bbfe428e36fc92202e39.dataidx
- $TESTTMP/hgcache/master/packs/8c654541e4f20141a894bbfe428e36fc92202e39.datapack
- $ hg cat -r . x
- x2
- $ hg cat -r . y
- y
- $ hg cat -r . z
- z
-
-# Prefetch packs that include renames
- $ cd ../master
- $ hg mv z z2
- $ hg commit -m 'move z -> z2'
- $ cd ../packprefetch
- $ hg pull -q
- (running background incremental repack)
- $ hg prefetch -r tip
- 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
- $ hg up tip -q
- $ hg log -f z2 -T '{desc}\n'
- move z -> z2
- x
# Revert across double renames. Note: the scary "abort", error is because
# https://bz.mercurial-scm.org/5419 .
+ $ cd ../master
+ $ hg mv z z2
+ $ hg commit -m 'move z -> z2'
+ $ cd ../shallow2
+ $ hg pull -q
$ clearcache
$ hg mv y y2
+ y2: not overwriting - file exists
+ ('hg rename --after' to record the rename)
+ [1]
$ hg mv x x2
+ x2: not overwriting - file exists
+ ('hg rename --after' to record the rename)
+ [1]
$ hg mv z2 z3
+ z2: not copying - file is not managed
+ abort: no files to copy
+ [255]
$ hg revert -a -r 1 || true
- forgetting x2
- forgetting y2
- forgetting z3
- adding z
- undeleting x
- undeleting y
- 3 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob)
+ 3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over * (glob)
abort: z2 at 109c3a557a73: not found in manifest! (?)
diff --git a/hgext/remotefilelog/shallowrepo.py b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -118,9 +118,6 @@
fileservicedatawrite = cachecontent
fileservicehistorywrite = cachemetadata
- if repo.ui.configbool('remotefilelog', 'fetchpacks'):
- fileservicedatawrite = packcontentstore
- fileservicehistorywrite = packmetadatastore
repo.fileservice.setstore(repo.contentstore, repo.metadatastore,
fileservicedatawrite, fileservicehistorywrite)
shallowutil.reportpackmetrics(repo.ui, 'filestore',
diff --git a/hgext/remotefilelog/fileserverclient.py b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -10,7 +10,6 @@
import hashlib
import io
import os
-import struct
import threading
import time
@@ -30,8 +29,6 @@
contentstore,
lz4wrapper,
metadatastore,
- shallowutil,
- wirepack,
)
_sshv1peer = sshpeer.sshv1peer
@@ -316,10 +313,6 @@
cache = self.remotecache
writedata = self.writedata
- if self.ui.configbool('remotefilelog', 'fetchpacks'):
- self.requestpack(fileids)
- return
-
repo = self.repo
count = len(fileids)
request = "get\n%d\n" % count
@@ -453,61 +446,6 @@
self.writedata.addremotefilelognode(filename, bin(node),
lz4wrapper.lz4decompress(data))
- def requestpack(self, fileids):
- """Requests the given file revisions from the server in a pack format.
-
- See `remotefilelogserver.getpack` for the file format.
- """
- try:
- with self._connect() as conn:
- total = len(fileids)
- rcvd = 0
-
- remote = conn.peer
- remote._callstream("getpackv1")
-
- self._sendpackrequest(remote, fileids)
-
- packpath = shallowutil.getcachepackpath(
- self.repo, constants.FILEPACK_CATEGORY)
- pipei = remote._pipei
- receiveddata, receivedhistory = wirepack.receivepack(
- self.repo.ui, pipei, packpath)
- rcvd = len(receiveddata)
-
- self.ui.log("remotefilefetchlog",
- "Success(pack)\n" if (rcvd==total) else "Fail(pack)\n",
- fetched_files = rcvd,
- total_to_fetch = total)
- except Exception:
- self.ui.log("remotefilefetchlog",
- "Fail(pack)\n",
- fetched_files = rcvd,
- total_to_fetch = total)
- raise
-
- def _sendpackrequest(self, remote, fileids):
- """Formats and writes the given fileids to the remote as part of a
- getpackv1 call.
- """
- # Sort the requests by name, so we receive requests in batches by name
- grouped = {}
- for filename, node in fileids:
- grouped.setdefault(filename, set()).add(node)
-
- # Issue request
- pipeo = remote._pipeo
- for filename, nodes in grouped.iteritems():
- filenamelen = struct.pack(constants.FILENAMESTRUCT, len(filename))
- countlen = struct.pack(constants.PACKREQUESTCOUNTSTRUCT, len(nodes))
- rawnodes = ''.join(bin(n) for n in nodes)
-
- pipeo.write('%s%s%s%s' % (filenamelen, filename, countlen,
- rawnodes))
- pipeo.flush()
- pipeo.write(struct.pack(constants.FILENAMESTRUCT, 0))
- pipeo.flush()
-
def connect(self):
if self.cacheprocess:
cmd = "%s %s" % (self.cacheprocess, self.writedata._path)
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -58,7 +58,6 @@
``remotefilelog.debug`` turn on remotefilelog-specific debug output
``remotefilelog.excludepattern`` pattern of files to exclude from pulls
``remotefilelog.includepattern``pattern of files to include in pulls
- ``remotefilelog.fetchpacks`` if set, fetch pre-packed files from the server
``remotefilelog.fetchwarning``: message to print when too many
single-file fetches occur
``remotefilelog.getfilesstep`` number of files to request in a single RPC
@@ -153,7 +152,6 @@
configitem('remotefilelog', 'cacheprocess.includepath', default=None)
configitem("remotefilelog", "cachelimit", default="1000 GB")
-configitem('remotefilelog', 'fetchpacks', default=False)
configitem('remotefilelog', 'fallbackpath', default=configitems.dynamicdefault,
alias=[('remotefilelog', 'fallbackrepo')])
To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
More information about the Mercurial-devel
mailing list