D6288: automation: shore up rebooting behavior
indygreg (Gregory Szorc)
phabricator at mercurial-scm.org
Fri Apr 19 10:41:40 EDT 2019
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
There was a race condition in the old code. Use
instance.stop()/instance.start() to eliminate it.
As part of debugging this, I also found another race condition
related to PowerShell permissions after the reboot. Unfortunately,
I'm not sure the best way to work around it. I've added a comment
diff --git a/contrib/automation/hgautomation/aws.py b/contrib/automation/hgautomation/aws.py
@@ -808,10 +808,26 @@
# Reboot so all updates are fully applied.
+ # We don't use instance.reboot() here because it is asynchronous and
+ # we don't know when exactly the instance has rebooted. It could take
+ # a while to stop and we may start trying to interact with the instance
+ # before it has rebooted.
print('rebooting instance %s' % instance.id)
+ 'Delay': 5,
+ # There is a race condition here between the User Data PS script running
+ # and us connecting to WinRM. This can manifest as
+ # "AuthorizationManager check failed" failures during run_powershell().
+ # TODO figure out a workaround.
print('waiting for Windows Remote Management to come back...')
client = wait_for_winrm(instance.public_ip_address, 'Administrator',
To: indygreg, #hg-reviewers
More information about the Mercurial-devel