fix: host_bindgen! returns Result instead of panicking on guest errors

[jsturtevant]

fixes: #1316

The host_bindgen! macro generated export wrapper functions that called panic! when Callable::call() returned Err (e.g., guest abort, trap, timeout). This crashed the host process instead of letting callers handle the error gracefully.

Breaking Change

Host-side trait implementations generated by host_bindgen! must now return Result<T, HyperlightError> instead of bare T. Existing implementations need to wrap return values in Ok(...). Constructors (fn new) are unaffected.

This more closely matches the non-wit function calls which have errors that are returned.

Signed-off-by: James Sturtevant jsturtevant@gmail.com

[ludfjig]

I think I am fine with this, but I suspect this is intentional. I think we generally want to avoid Result types as much as possible where it makes sense, but since this matches the current guest calling convention maybe let's defer that discussion to later

[jsturtevant]

I think I am fine with this, but I suspect this is intentional. I think we generally want to avoid Result types as much as possible where it makes sense, but since this matches the current guest calling convention maybe let's defer that discussion to later

Open to other ways of addressing this I guess but a panic in the guest shouldn't panic this host. It seemed practical to match existing behavoir

[syntactically]

I agree with @ludfjig that, generally speaking, the semantic gap of lifting everything from the interface into Option isn't particularly attractive. However, I do see the need to deal with errors that fundamentally can happen during execution on the host side.

Is there a reason that this is also applied to the host functions imported by the guest? I think that generally speaking it is the ability to export a function implementing an interface, but actually implement a different interface (i.e. one with more optionality) that is the most semantically damaging---so the implicit Results are particularly unattractive on guest->host function calls (and to some extent in the implementation of guest functions, although the ability to trap/panic/etc is perhaps equivalent). It seems like splitting this (so that host->guest function calls can return errors, but host functions can't be implemented by returning errors) is what is common in the prior art e.g. the wasmtime host bindgen. Is it just about continuing to be able to use the same trait for the import and export versions of an interface? Maybe a type parameter to track whether this is an imported or exported usage would make sense?

Let me know what you think about the usability of that. As I think about this more, from the wasm perspective especially, I do think that having the Results on host function calls is slightly unattractive, but it's not quite as bad as I would have guessed, since from wasm's perspective, a (wasm) import that's called always does in fact have the option of causing the entire world to be torn town via purposefully causing a trap, and this is perhaps in some ways similar behaviour.

[jsturtevant]

Is there a reason that this is also applied to the host functions imported by the guest?

This was to match the current non-wit implementation. We had improved the situation with errors in #868.

I think that generally speaking it is the ability to export a function implementing an interface, but actually implement a different interface (i.e. one with more optionality) that is the most semantically damaging---so the implicit Results are particularly unattractive on guest->host function calls (and to some extent in the implementation of guest functions, although the ability to trap/panic/etc is perhaps equivalent). It seems like splitting this (so that host->guest function calls can return errors, but host functions can't be implemented by returning errors) is what is common in the prior art e.g. the wasmtime host bindgen.

Agree with the general sentiment that's its not ideal. I am not so sure we don't want host function to return errors, it seems useful to me to be able to say something went wrong across the boundry. Be interesting in what others think

[jsturtevant]

Agree with the general sentiment that's its not ideal. I am not so sure we don't want host function to return errors, it seems useful to me to be able to say something went wrong across the boundry. Be interesting in what others think

actually, was thinking on this some more and WIT does have the ability capture "result" types so we probalby don't need this on the guest side (I don't see the equivalent ::std::result::Result::Ok(#ret) = #ret else { panic!("bad return from guest {:?}", #ret) }; on the Guest->Host calls. Maybe there is another way to handle that too... I am open to ideas just really think we shouldn't panic if guest traps.

[syntactically]

just really think we shouldn't panic if guest traps

I'm with you on that one :)

This was to match the current non-wit implementation. We had improved the situation with errors in #868.

I am not so sure we don't want host function to return errors, it seems useful to me to be able to say something went wrong across the boundry.

I'm still not totally convinced on this, although I can see the arguments. Can you elaborate a little more on the use cases you see? If we did allow this, I think it's important that it not be semantically visible in the guest, which shouldn't know whether its imported component is implemented across a hyperlight-host boundary or by any other component, so we would need to propagate the error return up, and I would think that it would probably make sense to poison the sandbox (probably by panicking the guest?). If there's not a super strong use case, having the optionality just on the host->guest calls and not the hostcall returns seems like it is a starting point that gets rid of the main issue and is minimally invasive?

actually, was thinking on this some more and WIT does have the ability capture "result" types so we probalby don't need this on the guest side (I don't see the equivalent ::std::result::Result::Ok(#ret) = #ret else { panic!("bad return from guest {:?}", #ret) }; on the Guest->Host calls. Maybe there is another way to handle that too...

There is an unwrap on the host call result here.

Precisely the thing that I want to be careful about here is conflating WIT-level semantically-there-in-the-API optionality with this unconditionally-added-everywhere Hyperlight-level optionality. As I said in the paragraph above, even if we did add this for host function return types, I would not want to see it meaningfully exposed to the guest code.

[jsturtevant]

I am not so sure we don't want host function to return errors, it seems useful to me to be able to say something went wrong across the boundry.

I'm still not totally convinced on this, although I can see the arguments. Can you elaborate a little more on the use cases you see? If we did allow this, I think it's important that it not be semantically visible in the guest, which shouldn't know whether its imported component is implemented across a hyperlight-host boundary or by any other component, so we would need to propagate the error return up, and I would think that it would probably make sense to poison the sandbox (probably by panicking the guest?). If there's not a super strong use case, having the optionality just on the host->guest calls and not the hostcall returns seems like it is a starting point that gets rid of the main issue and is minimally invasive?

Yes I am in agree that we shouldn't do this now. I'm going to revisit this and see if I can come up with something different for consideration.