Allow callers to modify a running instance's devices and backends

[gjcolombo]

This is needed to support hotplug but might also be used to update backend parameters at runtime (e.g. sending Crucible a replacement VolumeConstructionRequest).

This roughly corresponds to this TODO in instance_ensure:

propolis/bin/propolis-server/src/lib/server.rs

Lines 259 to 267 in d4bc251

	// If properties match, we return Ok. Otherwise, we could attempt to
	// update the instance.
	//
	// TODO: The initial implementation does not modify any properties,
	// but we plausibly could do so - need to work out which properties
	// can be changed without rebooting.
	//
	// TODO: We presumably would want to alter network interfaces here too.
	// Might require the instance to be powered off.

[gjcolombo]

With the advent of #470 and #495 this is going to become increasingly important. At a minimum, we need to design a mechanism to ensure that a VM's set of entities remains stable during a migration.

[luqmana]

I'd actually not limit this to just propolis-server but make it a part of the core propolis lib.

There's definitely some design work to do but I'm a bit inclined to have the inventory just store Arc<RwLock<dyn Entity>>'s. The general idea being, most access can be via shared refs but occasionally we'd like exclusive access to a particular entity. This does come with some difficulties in straddling async/sync contexts as well as ensuring fairness (readers shouldn't starve out writers).

[pfmooney]

have the inventory just store Arc<RwLock<dyn Entity>>'s.

That would be a pretty substantial change when it comes to the dyn types in propolis generally.

The general idea being, most access can be via shared refs but occasionally we'd like exclusive access to a particular entity.

What operations are you thinking would use such exclusive access to an entity?

[luqmana]

Yea, definitely not a simple tweak. Mostly just been thinking about modeling block backend/device interactions and allowing re-configuring a backend.