Skip to content

Add a newer Zstd module for Swift use, with a more idiomatic API, plus nullability annotations#4665

Open
wadetregaskis wants to merge 5 commits into
facebook:devfrom
wadetregaskis:dev
Open

Add a newer Zstd module for Swift use, with a more idiomatic API, plus nullability annotations#4665
wadetregaskis wants to merge 5 commits into
facebook:devfrom
wadetregaskis:dev

Conversation

@wadetregaskis
Copy link
Copy Markdown

This is a set of related patches which improve Swift support, by providing a new version of the Swift module with an improved API (named Zstd to be distinct from the existing libzstd which remains, unchanged, for backwards compatibility). This new version of the Swift API differs in that:

  • Prefixes like "ZSTD_" are removed (in Swift the module prefix may be used if necessary or preferred, e.g. Zstd.compress(…)).
  • Names are more idiomatic (e.g. ZSTD_inBuffer_sInputBuffer, ZSTD_getDictID_fromDDictdictionaryID(fromDecompressionDictionary:)).
  • Function arguments are labelled where appropriate (e.g. ZSTD_compresscompress(into:capacity:from:size:level:)).
  • Implicitly-unwrapped optionals are avoided (using new nullability annotations, which also apply to the C API).
  • Closed enums (those that aren't explicitly expected to add new cases) now import as native Swift enums, rather than just free-standing integer constants.

I'd really like to map the C structs to Swift structs / classes, but unfortunately in order to do so those C structs have to be defined in the headers. Currently they are merely declared in the headers, making them pseudo-anonymous. If changing that is an option, I'd be happy to do that (the mapping into Swift would still not be perfect, since these C structs aren't reference counted nor intended to be pure value types, so they don't map cleanly to Swift classes or structs - but that can be mitigated).

Note that the C API is changed in that the nullability annotations apply there too, under compatible compilers (e.g. Clang). This has limited if any effect at runtime (the nullability information is compiler metadata only, though it may influence optimisation decisions) but may introduce new compilation warnings or errors for users of the C API. Some of those errors or warnings may be false positives - e.g. cases where the code doesn't clearly prevent inappropriate use of null but in practice null never happens.

I tried to ensure libzstd is completely unchanged, including continued use of implicitly-unwrapped optionals - though I could go either way on that specific point. Since the C API is technically changing (in a limited way - nullability annotations), maybe it's reasonable to change the existing Swift API in that limited way too?

Alternatives considered

Do nothing

The existing API, as expoed to Swift in libzstd, is functional. But it's pretty unpleasant to use, because of how C works in Swift (it's a little like having to write assembly in C, in terms of appeal, complexity, and [lack of] safety). Granted this new Zstd Swift module doesn't entirely solve that - it's still dealing with raw pointers, for example - but it does help a lot.

Rely on intermediary Swift packages

There are existing 3rd party Swift packages which wrap the C library into an even more idiomatic Swift interface (with proper structs or classes, and extensions on standard Swift types like Data and [UInt8]), and it's certainly possible to implement changes like that here too - we can include C and/or Swift shims. I didn't pursue that [for now] because:

  1. That's a lot more work, to make a truly elegant Swift API.
  2. There's a lot more subjective API design decisions to be made there.
  3. It can always be done later; nothing in this set of patches precludes that.

This is generated (by default) when building with `swift` (the Swift CLI), e.g. `swift build`.
This folder contains generated Xcode project files that are used locally in some circumstances but should never be checked in (they're derived from Package.swift).
…existing `libzstd`.

The new module exposes the same fundamental API but with names that are cleaner and more idiomatic in Swift, and with some key additional Swift-specific annotations (such as on closed enums, to make them import as Swift enums rather than just free-floating constants).

`libzstd` remains unchanged for backwards compatibility (or those that prefer the "raw" C API, I suppose).

Note that this stops short of actual code changes or API changes beyond just renames (e.g. it does not change the parameter orders to be more idiomatic in Swift, nor present the various structs as actual Swift structs or classes).  It _could_ - it could include a whole shim layer, written in C and/or Swift, to make the Swift API look like whatever we want - but it's a conscious choice to not go that far at this time.  There are existing Swift packages which wrap this library and perform more substantial API changes.
These apply to the C API as well (but are not supported by all compilers, e.g. GCC still doesn't AFAIK).

They are important for Swift because they prevent the API using implicitly unwrapped optionals everywhere.  In many cases the parameter (or return value) is _not_ in fact optional, and when it is that should be exposed explicitly in Swift (per Swift best practice).
@meta-cla meta-cla Bot added the CLA Signed label May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant