As part of mass cloning repositories on hg.mozilla.org, I noticed that various repositories differed by only their phases state. Upon further examination, it appears I found a bug where `hg pull` results in local changesets being in the draft phase when they are public on the server! I'm still working on STR, but I have an example from hg.mozilla.org. The server is running Mercurial 4.5.3 with devel.legacy.exchange=phases and the issue reproduces on clients from @ down to at least 4.3.3. The hg.mozilla.org/users/gszorc_mozilla.com/python-zstandard repo reproduces the problem near the tip of the repository. On the canonical SSH server (where people push to): o 67d74 draft news: fix typo | o 314a0 draft Merge 0.9 branch into main branch |/| | o 54909 public Added tag 0.9.1 for changeset f159317288f3 | o f1593 public news: release 0.9.1 | o f6a7a public global: bump version to 0.9.1 in preparation for release | o 9de87 public ci: use Python 3.6.5 in AppVeyor | o 1f91a public ci: add extra hash for pluggy | o 1ffac public setup: fix typo in `remove()` call | o eaf7d public setup: support building with mingw compiler | o 17b41 public debian: add Debian packaging | o cd231 public ci: pin pip and virtualenv versions in appveyor o | ca940 draft ci: add extra hash for pluggy o | c7286 draft bench: add support for long distance matching o | bfebc draft bench: use ZstdCompressionParameters o | 44ee4 draft global: bump version to 0.10.0.dev0 |/ o 1172a public Added tag 0.9.0 for changeset 02d538f2b88a And on the client (including the https://hg.mozilla.org/ servers, which `hg pull ssh://hg.mozilla.org/` under the hood): o 67d74 draft news: fix typo | o 314a0 draft Merge 0.9 branch into main branch |/| | o 54909 draft Added tag 0.9.1 for changeset f159317288f3 | o f1593 draft news: release 0.9.1 | o f6a7a draft global: bump version to 0.9.1 in preparation for release | o 9de87 public ci: use Python 3.6.5 in AppVeyor | o 1f91a public ci: add extra hash for pluggy | o 1ffac public setup: fix typo in `remove()` call | o eaf7d public setup: support building with mingw compiler | o 17b41 public debian: add Debian packaging | o cd231 public ci: pin pip and virtualenv versions in appveyor o | ca940 draft ci: add extra hash for pluggy o | c7286 draft bench: add support for long distance matching o | bfebc draft bench: use ZstdCompressionParameters o | 44ee4 draft global: bump version to 0.10.0.dev0 |/ o 1172a public Added tag 0.9.0 for changeset 02d538f2b88a Here, changesets f6a7a::54909 are incorrectly draft instead of public. If you diff the .hg/store/phaseroots file, we find that the client has an extra entry: 1 f6a7a8e48af15f90f456a842e09ac2fefa724b65 If I ssh into the server and manually issue a "listkeys" wire protocol command for the "phases" namespace: $ ssh hg.mozilla.org hg -R users/gszorc_mozilla.com/python-zstandard serve --stdio listkeys namespace 6 phases 7137 986f569345bba5c6eb263e050f785f3e6f1bbc25 1 ec4c657a63e5f6c5aacdc28c6bc39c3760846813 1 811f1d2192ea34458e838cd88f5a771617405cd4 1 c69432dce70a19e403e46853a79427be6d2f27b4 1 a6984f3d05b40ceaa4ba494890e2452d0bef56da 1 3d9788d3463bd0547eb3e91f2157133fadf77297 1 49b9a0625aee45b74904496730dd82baa514e13a 1 d1da1bf1d557cd67d48abd23e564e4a4c824ddb2 1 4aeb2f3f32ae051cf388aece928818af51eca53f 1 4dde54e76679b34ebeb5ca529ff260bac278382d 1 95899a27c27aa87ead26b76e8a81d898526ea21e 1 56f38907e7d3957fb627f4653ced21a1b140e519 1 b19288f3a97716bab227757c244a406a98fa2851 1 7f3d120cd235fce70544a7c7c08724a25f7517da 1 1db425d434de9241f95b05657d85b1f0a3a77fb6 1 ed48d36f8266ce8c2443e081b1b5e5b4262dad03 1 69f6c636f84a23b05b38dcabb501d50dce66f0fb 1 0b3f3f44b473dfd05160386372203818b00dbf19 1 6b7eb8703a3fe547e0be7300565177c6aea5c60d 1 309f42976e5399968b826a4c19965905984fefea 1 fa4170581204af807885dbdbe8486acdf40af507 1 b09f702c49efd5e7e0df7d2890881830d3e5b2bc 1 e7275945ed75f58c149967fe9d88cb12c565f8de 1 2855d56e4a958568ea4b7d510983f97be867b0d9 1 6c38f8081563b66866b074239c4cbe119ff497e7 1 5247d6b44652dac7f6c176260aeca6163fd88cdb 1 0c99eed91292d2f274b70166e405c2a5feddb33b 1 86d2cb07d6f54e513e3f1a423a51ce72930d58cd 1 d13fd6b2f8f1c6741af515e88a95938c627ed015 1 116423c87db954323b5aee9fdc672290c46e72ff 1 6826d1a901995eb06d87d1eaecff1b16a3edec43 1 c6856ab2022261311881ecf7c2abde2e6f0df242 1 56e4af70d1fc332bef6599984b3a1e7a621ef3b3 1 d0ae76ac5dcfccee8e21ab16ca661c1cc70d5675 1 ee4a0a78069f6f9df95ebb8310791258f45f2776 1 f015adba1b7be454276977c6e9ba3f89649c0093 1 cabc428e2a44a6a9a00173c2f20e4d51d3545f0f 1 d413b84d36ae49c8637853134d9c8bec5f331769 1 b7c5a0705ece825d0ab9b0ac32c671757fabdc3c 1 e0d0a442155031c5d31bff18de2deddf10dcbb8f 1 eb0fb3f00d01e0e58cef6a07f5ecfc3c0f9e2480 1 274a11e5ee778e40c3b2b6d2a2735e7ee3d6ca16 1 2531802ea39dd7ef43c36937e79e400d446031d8 1 74c3941f6b719c940a668fdfecd1d48310c7692c 1 d08a192fe09315c640d5cc4b9e89bfefcc3d8b77 1 02cabf42f6831b8553ae60f3ad531dcdbbf6164e 1 d6c2f0c32f505be06f28009e07820ab98ac38de1 1 e41c573c5e40bf8688c14674ee4fe432e3680e53 1 8e600d2273a62ed2d4af1062142b3aa6183cac17 1 62098061791e987c8a3028e543111fd76e3eb5a2 1 358f141961556d713f71db1dd53a88a88ab8a99e 1 f5d32891cd9d4d52643a5369fffb0880900cc052 1 c4e75e3bb37291c6ffb39544df0a066d4c161b54 1 9a12cd0c44b07c0255dac713011b2ba8096bac6c 1 1c272833d33b62f06b9aeea23791cecd9bcc3fdc 1 dd16d69fa87802da0bca5578bbbd7f729f361adf 1 b8745313f84b3478dcb7e342ebfc0744ee1bafcf 1 8259a4b5f445ee4963b1fae3f91b1b1705dd5dce 1 76a725fface46755db64f524ed3319b271e17f90 1 a947f7b9c8a8135703022b35a8b9e6982a556f02 1 2e28c0d579de5d4df539ba9b92afd67b78d1ec0a 1 2dc893cb09716096ead864ca3b40f921d691f5f0 1 ff6f423ad49b0750ff9911fc3a024b526fd3b49b 1 b73ef61ed0725d1263e4e5966d0d47b11672f032 1 f61a8b7a2f57b1cff9fa2710aac2785859b62469 1 e3326137f62969d73186b13c3442563f6e8d50b0 1 8d61587e1527491178e31b5832dfa48353386ac5 1 24c42456ef34a00fa3e2b655ec7a833398e3923c 1 60ec011c69bee418a895a853de39fec5e0b79889 1 8e29f806d139ad1710805f15ba0ced97698eb642 1 c4a25b03802f818a97fd20940e13d09071cadba5 1 38ae0635a38fae170a0dfc3e09d6f5db39204bab 1 c6f2b88fb64c2ccaa1e75fb83eef534d40cf2f40 1 34e0004d10108bb6c0f34f2e10f24a01f2eb6a83 1 9409410d936e64a8aee1b361309ec7072ea67ab9 1 d8876f131a5e1c3a0cec26775bafa283eed42f33 1 0a339dfa1c83d55f78c75340a633ba0e9ed0d6c0 1 f89f68b4b897d8f4b83c7c67dc1edeca0b0114f5 1 fc9204b3bf229a8b0b635f6b82cd176ad8bd3935 1 3ff16b370b9fa27236b28ff31df5175f09d95c93 1 77abf93ff7cc62598f2e5c2d8ab8b00c0e28a375 1 2f62d010ffcc8de6696680f7a26b58ed2cba7167 1 e6b39e00a57a9e80a55d8b7826b05d00f17157e5 1 0d87be69fd94668ccd640071e6a6ab0a1e22f1e3 1 9d282f9c573f5078c3ffd5370554ea3845f158ef 1 017e76d051328213ffe4b3c3db265da3db8c0a1b 1 322ecc9d85003431a4df7171c3b6b9ba532b0047 1 aaa2b3154c3c191071abeff4bdf1d43e69fbf23b 1 7d64d9ebb7e8bdd5577183ad623eb71dd95a2655 1 ec0a33f968e77a7118a5e2d0a393310647b22197 1 9dadeac76dddda24f70ea20c2bd0171f8cd68361 1 32dff3dcf4294e82bf6ffa5b97a92d0377761d37 1 c805034df9cfc8458238b62cbd4b5d3dc90bbe94 1 67129d35448db515dc5d6f05c743a25cb909cee8 1 3b53040c4fe61b341359a58118c17d7e5b435c0c 1 ba1c60bfe2efabb2bb8131ecb6c29da0d5c99215 1 f0d59f2cee46374b2bf46d49146c08fef69f2567 1 d828011e57727e2e52de65c731c4aa5e4654f352 1 2a04b8dda45e706d1403413109be66005756e0f3 1 999c5f49207948ce21c5c5c3029fd59311192a5f 1 84947e2f8e98100ca72d1f1ccd37a1cbca038be5 1 02af48fdbcaa5b19d58376ec57d469f2c0cc384c 1 44ee411a7d63de3f148f1fd1762e5dfc9bcf5dc2 1 f29e3ddb4baeeae45040ca9bc99bbc4958446466 1 c2898329d636cc94b64fee2f704b7c1120642e96 1 5a03226a659a8a9a1546b9db02f4c55da0bb73de 1 067a095a9149b419a45a3fe2a4097ee1d0a1c97d 1 a70532c64c10e85e43aa27fb7fbc64c281984254 1 8a34c78c649e40b4224a81d5597b38b1884d513d 1 aea9fb2f58514e167ca55f099491133ac0be0c70 1 8ca5d5f84a204efa8c8817ff0e9e99d1eedbaf2f 1 71da8f870b1e4fa13c11831f38b16e5320d0fedb 1 ffcd2c9e09c3b36a31192aa30425b4a0b483324a 1 af01b2f6d7c4a0a8864e583da67d9bcf4732f7f1 1 0097b23f5791dd320b7f44449c5666c87b665cf5 1 1b1a27772d6933e261cd4428c5ada7df5c39de50 1 c4f04519003172f49e18bb7ca6d92dd627d5e895 1 ddc76723591dec064433069e98e77058753212e6 1 b287b687f05d2dbae6784904eaa438ea2e8d8dbe 1 460dcde1a47d47809d202dd16d8eddcefebbf5de 1 f817d6fc0a16603501a9e6ddef088abdf40611dd 1 44534d2f22cd2115a4728b5dfccbe4259d50670c 1 f523c4cea3cdb9eae7f0c92cca26465f8ce72720 1 ce4eb39edb26220a4fdfd14e8a9435542e209128 1 be7c94ed7df15acbab8949a99f2b99cced178d33 1 f3d5d68233e82d944651f7ce4e7862dd72b485f2 1 e1a965bd5026f06cf393b725b9796140294f4f9e 1 7b8b717f11e7428f64e6c5c92ab4d1bac56b0012 1 df24a3db03e325a2a7937f488916833c5fd2f6dc 1 06a626ab5bb583faf24451c7b3710119980013d1 1 6b36d4f5e5eaa8484f098ecdf3a6b781e6801db5 1 10d4212d38d1cb84ed4aa87ccf71f3bbd6a048f2 1 165a4ad238be35a256e7fe7d7961ef5ef3fbd544 1 6aa1592abc4ac1572442d83576eafc316e87e4b4 1 06438af7e4055a1cd37b5a8a0ae11c634886fe56 1 862968a771ab02775ed3bcb01c40a1f96fbcc181 1 2801b56230c2661e0eb378f85b1da99b041a4f1e 1 78022e104369a41f10195c09361c1f73388a1dfe 1 43f456430d97012373740663352ace35b5c19a6a 1 7b5d9a6fb9a4af3df38d88eeaf5a75c509a6bccb 1 ba90f4bc93904f4d65158abff074d4a60c7bc21b 1 4fbd99de09d2a67b4d3c5efdd6774148b47e1e24 1 e833e55b7c5ab82f41d7e5a446e2094526d55a46 1 4918e74778ea7f7da2f33a5af554a49ea2671083 1 44093aa6852d2743b11dfc783ce6b51cfcd81ab8 1 e8b3a3c42308527ad245e22abe6aa4405e3695de 1 35637b23cd45f29b650f5584dc3a087139402c1d 1 314f73d4093c257f7d9dfa9b932a36dd3778f64e 1 f35f457f10a0e6ce4895280501a7cc6b2b856316 1 ed3c4aac8611f0afecdb7ad3e10807b0ffdbcddc 1 c1b65f40f7737a5545c12d026782538d5c00ec86 1 7fb0c208f592cd6aa659090061a01967029253a3 1 3f5a762bf0a951bdee155f979e9c5fc82c215794 1 c3c39d39bceae7b628ef94ded4e4150509366f74 1 b1b75903162514065d7bdca7207d534eb8bdb424 1 92e93907aceb11a8e86473edc0d6ee4ad704fc28 1 70ebcd52447f4553c6d38e6641f725714c3b065b 1 e03ac7083fc53863296c3710e64b90cb06832231 1 58f2f007cc9900fd814be6020d3f29c802887715 1 d0e33dec575bfd99fff15be02eadf4d7956e6c99 1 ecc791c16e683e20a5d027d83e42418915bea718 1 d05d95690ef7fdbf0d5bb4c494d30a2ede251af6 1 e50d338ec965ee88dc1922e485be426222ff09d9 1 78f3a23ab6b1559c8831c894c3ee31d97236393d 1 331ccd8fd7715f029616198f85f0fd71be257fa8 1 7e2d01f37df8c737783767bb892a6ad238a564ad 1 The f6a7a8e48af15f90f456a842e09ac2fefa724b65 entry is missing. That's good. But there are a number of heads in this repo. Let's look at a repo with fewer heads... The hg.mozilla.org/integration/autoland repo has a single head. But it still exhibits this problem. On the canonical server: o b099e7 draft Bug 1454358, removes unneccessary implementation of ScrollBoxObject rr?enndeakin+6102 r=bz,enndeakin+6102 o 4aa9ef draft Merge mozilla-central to autoland. a=merge CLOSED TREE |\ | o 3aca10 public Merge inbound to mozilla-central. a=merge | |\ | | o 4f5a3c public Backed out 1 changesets (bug 1473637) for frequent wpt failures inputevent-constructor.htm/window-named-properties-002.html CLOSED TREE | | o b60d99 public Bug 1474306 - Fix typo in the extension optionsType handler. r=Mossop | | o 25915c public Backed out 2 changesets (bug 1474569, bug 1474581) for wpt failures in test harness output on a CLOSED TREE | | o b39da1 public Bug 1474581 - Add support for external scripts in wpt .any.js tests in jstests harness; r=jgraham | | o d8c824 public Bug 1474569 - Improve output for failing wpt tests in jstests harness; r=bbouvier | | o 853cf1 public Bug 1470447 - JPEG decoder should post an invalidation for each row. r=tnikkel | | o 7fb226 public Bug 1464910 [wpt PR 11204] - Update wpt metadata, a=testonly | | o 64cceb public Bug 1464910 [wpt PR 11204] - Split up WebCryptoAPI/derive_bits_keys/ with `variant`, a=testonly ... | | o e41340 public Backed out changeset e711420b85f7 (bug 1461793) for causing bug 1474538 | | o e7824e public Merge mozilla-central to inbound. a=merge CLOSED TREE | | |\ | | | o 3978561 public Bug 1474522 - Change PrepareScriptEnvironmentAndInvoke to always take a global object instead of an arbitrary scope. r=bho lley | | | o e378e2 public Bug 1468536 - Remove Debugger.Object.prototype.global. r=jimb ... | | | o 1dbf99 public Bug 1464783 [wpt PR 11177] - Update wpt metadata, a=testonly | | | o 5e00e2 public Bug 1464783 [wpt PR 11177] - webrtc wpt: remove test_state_change_event, a=testonly o | | | 7ef930 draft Bug 1474898 - Fix typo in null-check. r=asuth o | | | ecafdb draft Bug 1423011 - Part 4: Fix for overflow-x:hidden test failures in the mochitest test_wheel_default_action.html. r=botond o | | | 3dc215 draft Bug 1423011 - Part 3: Add mochitests. r=botond ... o | | | a6dc6d draft Bug 1474721 - Drop epoch times in nsRefreshDriver. r=bz o | | | db1c00 draft Bug 1452348 - disable intermittent test on android + debug r=mixedpuppy o | | | 03a018 draft Bug 1474786 - Update Fluent to master. r=stas |/ / / o | | 4e729c public Bug 1474638 - Remove nsIDownloadHistory::removeAllDownloads. r=mak o | | 43dc23 public Bug 1474638 - Change users of nsIDownloadHistory::removeAllDownloads() to PlacesUtils.history.removeVisitsByFilter. r=mak o | | 8de2dc public Bug 1463938 - Recalculate frecency in chunks on idle. r=mak And on someone who has `hg pull`d it: o b099e7 draft Bug 1454358, removes unneccessary implementation of ScrollBoxObject rr?enndeakin+6102 r=bz,enndeakin+6102 o 4aa9ef draft Merge mozilla-central to autoland. a=merge CLOSED TREE |\ | o 3aca10 draft Merge inbound to mozilla-central. a=merge | |\ | | o 4f5a3c draft Backed out 1 changesets (bug 1473637) for frequent wpt failures inputevent-constructor.htm/window-named-prope | | o b60d99 draft Bug 1474306 - Fix typo in the extension optionsType handler. r=Mossop | | o 25915c draft Backed out 2 changesets (bug 1474569, bug 1474581) for wpt failures in test harness output on a CLOSED TREE | | o b39da1 draft Bug 1474581 - Add support for external scripts in wpt .any.js tests in jstests harness; r=jgraham | | o d8c824 draft Bug 1474569 - Improve output for failing wpt tests in jstests harness; r=bbouvier | | o 853cf1 draft Bug 1470447 - JPEG decoder should post an invalidation for each row. r=tnikkel | | o 7fb226 draft Bug 1464910 [wpt PR 11204] - Update wpt metadata, a=testonly | | o 64cceb draft Bug 1464910 [wpt PR 11204] - Split up WebCryptoAPI/derive_bits_keys/ with `variant`, a=testonly ... | | o e41340 draft Backed out changeset e711420b85f7 (bug 1461793) for causing bug 1474538 | | o e7824e draft Merge mozilla-central to inbound. a=merge CLOSED TREE | | |\ | | | o 3978561 draft Bug 1474522 - Change PrepareScriptEnvironmentAndInvoke to always take a global object instead of an arbitr | | | o e378e2 draft Bug 1468536 - Remove Debugger.Object.prototype.global. r=jimb ... | | | o 1dbf99 draft Bug 1464783 [wpt PR 11177] - Update wpt metadata, a=testonly | | | o 5e00e2 draft Bug 1464783 [wpt PR 11177] - webrtc wpt: remove test_state_change_event, a=testonly o | | | 7ef930 draft Bug 1474898 - Fix typo in null-check. r=asuth o | | | ecafdb draft Bug 1423011 - Part 4: Fix for overflow-x:hidden test failures in the mochitest test_wheel_default_action.ht o | | | 3dc215 draft Bug 1423011 - Part 3: Add mochitests. r=botond ... o | | | a6dc6d draft Bug 1474721 - Drop epoch times in nsRefreshDriver. r=bz o | | | db1c00 draft Bug 1452348 - disable intermittent test on android + debug r=mixedpuppy o | | | 03a018 draft Bug 1474786 - Update Fluent to master. r=stas |/ / / o | | 4e729c public Bug 1474638 - Remove nsIDownloadHistory::removeAllDownloads. r=mak o | | 43dc23 public Bug 1474638 - Change users of nsIDownloadHistory::removeAllDownloads() to PlacesUtils.history.removeVisitsBy o | | 8de2dc public Bug 1463938 - Recalculate frecency in chunks on idle. r=mak The discrepancy here follows a similar pattern: a DAG branch visually on the right side is draft instead of public. Here, 5e00e2::3aca10. If you diff the .hg/store/phaseroots files, the client has an extra entry: 1 5e00e20d858a484df71758c63a873aaa52483c21 If we ask the server for phase roots via the wire protocol: $ ssh hg.mozilla.org hg -R integration/autoland serve --stdio listkeys namespace 6 phases 85 03a018872c480ac6a43675a88c58439fdaf808e6 1 c98c89a11df28e970c975088f1cfefc847aa06eb 1 This matches the content of .hg/store/phaseroots on the server. So all looks good here. For context, c98c89a11df28e970c975088f1cfefc847aa06eb is a hidden changeset. 03a018872c480ac6a43675a88c58439fdaf808e6 is the only visible phaseroot. So it kinda looks like there is a bug in our phase advancement logic. Possibly around not traversing all ancestors when we encounter a phaseroot that is a merge.
I've isolated this to devel.legacy.exchange=phases. I /think/ that means that the issue will reproduce when an old client connects to a modern server. So this is still worth fixing.
Test added at https://phab.mercurial-scm.org/D3932. I'll repeat what I said there: I don't understand phases code that well and don't think I'll have time to look at this in the next few days. If someone else could help out with a patch, it would be appreciated. The bug is somewhere in exchange._pullapplyphases() or the code it calls.
Fixed by https://mercurial-scm.org/repo/hg/rev/2a227782e754 Gregory Szorc <gregory.szorc@gmail.com> tests: add test demonstrating phase loss when cloning (issue5939) The added tests demonstrate that phases exchange when using the listkeys based phases exchange fails to preserve public phase in a certain scenario when a merge is a phase root. Both non-bundle2 and bundle2 prior to the binary phase data part are buggy. Differential Revision: https://phab.mercurial-scm.org/D3932 (please test the fix)
Fixed by https://mercurial-scm.org/repo/hg/rev/88efb7d6bcb6 Yuya Nishihara <yuya@tcha.org> phases: remove excessive optimization from newheads() (issue5939) This function is intended to compute 'heads(::heads - roots::)', but it failed because 'heads + parents(roots)' missed sibling branches of the roots. That's why the public heads slipped down from D to B in the example added by 2a227782e754 "tests: add test demonstrating phase loss when cloning": > E draft > |\Z draft > | Y draft > D | public > | X draft > C/ public > B public > A public where heads = {E, Z}, roots = {X} (please test the fix)
Bug was set to TESTING for 7 days, resolving
Fixed by https://mercurial-scm.org/repo/hg/rev/f736fdbe546a Boris Feld <boris.feld@octobus.net> remotephase: avoid full changelog iteration (issue5964) Changeset 88efb7d6bcb6 introduced a performance regression by triggering a full ancestors walk. This changeset reworks this logic so that we no longer walk down the full changelog. The motivation for 88efb7d6bcb6, issue5939, is still fixed. mercurial compared to a draft repository ---------------------------------------- 8eeed92475d5: 0.012637 seconds 88efb7d6bcb6: 0.202699 seconds (x16) 46da52f4b820: 0.215551 seconds (+6%) this code: 0.008397 seconds (-33% from base) The payload size reduction we see in `test-bookmarks-pushpull.t` comes from a more aggressive filter of nullid and is harmless. (please test the fix)