feat: add external byte codecs

[andreiltd]

See HIP document for motivation for those changes: #1791

This PR provides the codec and API groundwork for the optimization. Legacy stack based host-guest communication continues to embed byte payloads in flatbuffers. The patch

• adds external value sources and sinks to function call and result codecs,
• keeps embedded codecs available for the existing stack transport,
• exposes ByteChunks parameters and typed guest returns through the C API.

[Copilot]

🟡 Changes recommended

The updated C guest path introduces a confirmed malloc leak and the new C API exposes an owned string without a matching deallocator, both of which must be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR lays the schema, codec, and C API groundwork for carrying byte payloads outside FlatBuffers, while preserving embedded byte encoding for the legacy stack-based transport.

Changes:

• Add ByteChunks as a distinct logical byte type, plus FlatBuffer compatibility representations for embedded transport.
• Introduce external byte value sources/sinks for function call and result codecs (encode markers in FlatBuffers, stream payloads out-of-band).
• Update the guest C API to use typed hl_ReturnValue* results and expose ByteChunks parameters and host returns.

File summaries

File	Description
src/tests/rust_guests/simpleguest/src/main.rs	Extend fuzz guest to encode ByteChunks results.
src/tests/c_guests/c_simpleguest/main.c	Update C guest to return typed hl_ReturnValue* and use new constructors.
src/schema/function_types.fbs	Add FlatBuffer tables/unions/enums for external bytes + chunked bytes.
src/hyperlight_host/src/sandbox/snapshot/file/config.rs	Add ByteChunks support to snapshot JSON type mirroring + tests.
src/hyperlight_host/src/func/mod.rs	Re-export Bytes for chunk-preserving byte values.
src/hyperlight_guest_capi/src/types/vec.rs	Add borrowed slice constructor; simplify copy_to_vec implementation contract.
src/hyperlight_guest_capi/src/types/return_value.rs	New owned FfiReturnValue representation with tag + union + ownership.
src/hyperlight_guest_capi/src/types/parameter.rs	Add ByteChunks parameter support and switch to owner-backed borrowed views.
src/hyperlight_guest_capi/src/types/function_call.rs	Add owner-backed OwnedFfiFunctionCall to keep borrowed pointers valid.
src/hyperlight_guest_capi/src/types/byte_chunks.rs	New C-ABI types for borrowed/owned chunk descriptors and ownership helpers.
src/hyperlight_guest_capi/src/types.rs	Export new byte-chunk and return-value types.
src/hyperlight_guest_capi/src/return_value.rs	New C API entrypoints for constructing/reading typed return values.
src/hyperlight_guest_capi/src/lib.rs	Replace flatbuffer module export with return_value.
src/hyperlight_guest_capi/src/flatbuffer.rs	Remove legacy “flatbuffer result” C API helpers.
src/hyperlight_guest_capi/src/dispatch.rs	Update C guest dispatch to consume hl_ReturnValue* and encode via FlatBuffers.
src/hyperlight_guest_capi/README.md	Document ByteChunks semantics and the new hl_result_from_* contract.
src/hyperlight_guest_capi/include/macro.h	Update wrappers to return hl_ReturnValue* via hl_result_from_*.
src/hyperlight_guest_capi/cbindgen.toml	Rename/exports for ReturnValue and ByteChunks C-visible types.
src/hyperlight_guest_bin/src/guest_function/definition.rs	Add embedded encoding support for ReturnValue::ByteChunks.
src/hyperlight_component_util/src/hl.rs	Fix explicit Vec::<u8>::new() emission for empty results.
src/hyperlight_common/src/func/ret_type.rs	Add Vec<Bytes> as supported return type + round-trip test.
src/hyperlight_common/src/func/param_type.rs	Add Vec<Bytes> as supported parameter type + round-trip test.
src/hyperlight_common/src/func/mod.rs	Re-export Bytes from common func module.
src/hyperlight_common/src/flatbuffers/mod.rs	Wire new generated FlatBuffer modules into the tree.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_value_generated.rs	Regenerate union discriminants for new return value variants.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_value_box_generated.rs	Regenerate union accessors/verifier for new return value variants.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/return_type_generated.rs	Regenerate return type enum for hlbytechunks.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_value_generated.rs	Regenerate parameter value union discriminants for external/chunked bytes.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_type_generated.rs	Regenerate parameter type enum for hlbytechunks.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/parameter_generated.rs	Regenerate parameter union accessors/verifier for external/chunked bytes.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlsizeprefixedbytechunks_generated.rs	New generated table for embedded chunked returns.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlexternalbytes_generated.rs	New generated marker table for external byte payloads.
src/hyperlight_common/src/flatbuffers/hyperlight/generated/hlbytechunks_generated.rs	New generated table for embedded chunked parameters.
src/hyperlight_common/src/flatbuffer_wrappers/util.rs	Add byte-chunk utilities + FlatBuffer serialization for chunked returns.
src/hyperlight_common/src/flatbuffer_wrappers/mod.rs	Export new external codec traits module.
src/hyperlight_common/src/flatbuffer_wrappers/function_types.rs	Add ByteChunks value/type support + external encode/decode for results.
src/hyperlight_common/src/flatbuffer_wrappers/function_call.rs	Add external encode/decode for byte parameters + embedded ByteChunks support.
src/hyperlight_common/src/flatbuffer_wrappers/codec.rs	New ExternalValueSink / ExternalValueSource traits.
CHANGELOG.md	Document new C API return model and ByteChunks exposure.

Review details

• Files reviewed: 30/39 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ludfjig]

this looks great

[yoshuawuyts]

left some comments; I've reached out to @andreiltd offline to talk these through because that might be quickest

[yoshuawuyts]

@andreiltd and I had an offline chat; I now much better understand the decisions made in this PR. Resolved all my comments and happy for us to proceed with this!

[andreiltd]

I think copilot comments about leaks are technically correct, although they are not introduced by this PR. This code has been moved so they appear as added. I would prefer fixing it in a separate PR: expose paired dealloc function so that c users don't use free() for rust allocated content.

[jsturtevant]

@andreiltd and I had an offline chat; I now much better understand the decisions made in this PR. Resolved all my comments and happy for us to proceed with this!

would you put the understanding in the comments? I am interested in hear the response

[andreiltd]

Hey @jsturtevant, I added comments to Yosh's original questions. Let me know if that clarifies things a bit.