HomePhorge

When a Drydock Blueprint promises us a resource but can't deliver, continue…

Description

When a Drydock Blueprint promises us a resource but can't deliver, continue believing in it

Summary:
Ref T13073. When a Blueprint says it will be able to allocate a resource but then throws an exception while attempting that allocation, we currently fail the lease permanently.

This is excessively harsh. This blueprint may have the best of intentions and have encountered a legitimately unforseeable failure (like a vm.new call to build a VM failed) and be able to succeed in the future.

Even if this blueprint is a dirty liar, other blueprints (or existing resources) may be able to satisfy the lease in the future.

Even if every blueprint is implemented incorrectly, leaving the lease alive lets it converge to success after the blueprints are fixed.

Instead of failing, log the issue and yield.

(In the future, it might make sense to distinguish more narrowly between "actually, all the resources are used up" and all other failure types, since the former is likely more routine and less concerning.)

Test Plan:

  • Wrote a broken Hoax blueprint which always claims it can allocate but never actually allocates (just throw in allocateResource()).
  • Used bin/phd drydock lease to acquire a Hoax lease.
  • Before patch: lease abruptly failed permanently.
  • After patch: lease yields after allocation fails.

{F5427747}

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T13073

Differential Revision: https://secure.phabricator.com/D19070

Details

Provenance
epriestleyAuthored on Feb 13 2018, 3:08 AM
themackabuPushed on Mar 25 2025, 8:07 PM
Parents
rP6a4d5ce3c93e: Don't allow Herald Diff rules to "Call Webhooks"
Branches
Unknown
Tags
Unknown

Event Timeline