repr(ordered_fields)

[RustyYato]

View all comments

Add repr(ordered_fields) and provide a migration path to switch users from repr(C) to repr(ordered_fields), then change the meaning of repr(C) in the next edition.

This RFC is meant to be an MVP, and any extensions (for example, adding more reprs) are not in scope. This is done to make it as easy as possible to accept this RFC and make progress on the issue of repr(C) serving two opposing roles.

Rendered

To avoid endless bikeshedding, I'll make a poll if this RFC is accepted with all the potential names for the new repr. If you have a new name, I'll add it to the list of names in the unresolved questions section, and will include it in the poll.

[clarfonthey]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

[Lokathor]

Just as a small point of style the Guide Level Explanation is usually "what would be written in the rust tutorial book", and the Reference Level Explanation is "what could be written into the Rust Reference". This isn't a strict requirement, but personally I'd like to see the Reference Level part written out. Using the present tense, as if the RFC was accepted and implemented.

[programmerjake]

Suggested change

	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in memory layout.
	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in-memory layout.

"cross-platform" -- the layout will differ when there are different layouts for struct members' types, in particular primitive types can have different alignments which changes the amount of padding.

e.g., #[repr(ordered_fields)] struct S(u8, f64); doesn't have the same layout on x86_64 and i686

[RustyYato]

Good point, this will need to be documented as a hazard in the ordered_fields docs. However, the repr itself will be cross-platform. For example, #[repr(ordered_fields)] struct Cross([u8; 3], SomeEnum); will be truly cross-platform (given that SomeEnum is!).

[moonheart08]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

Just voicing support for repr(consistent) as naming.
Aside from the above, it more clearly hones in on the primary promises of the RFC, which is not just ordering but also exact type representation for things like enums. Field ordering is not the only thing it promises.

[joshtriplett]

Nominating this so that we can do a preliminary vibe-check on it in a lang triage meeting.

[scottmcm]

Big fan of doing this split, especially for structs. (It's less obvious what choices to make for other things, IMHO, but at least for structs this is something I've wanted for ages, so that for example Layout::extend can talk about it instead of C.)

Pondering the bikeshed: declaration_order or something could also be used to directly say what you're getting.

(This could be contrasted with other potential reprs that I wouldn't expect this RFC to add, but could consider as future work, like a deterministic_by_size_and_alignment where some restricted set of optimizations are allowed but you can be sure that usize and NonNull<String> can be mixed between different types while still getting the "same" field offsets, for example.)

[RustyYato]

I think this is also useful for unions, so we don't need to rely on repr(C) to ensure that all fields of a union are at offset 0.

This could be contrasted with other potential reprs that I wouldn't expect this RFC to add...

This also works as an argument against names like repr(consistent), since there are multiple consistent and useful repr, making it not descriptive enough.

[scottmcm]

I do think that declaration_order or ordered_fields is a bit weird on a union, because of course they're not really in any "order".

It makes me ponder whether we should just have repr(offset_zero) for unions to be explicit about it, or something.

(Which makes me think of other things like addressing rust-lang/unsafe-code-guidelines#494 by having a different constructs for "bag of maybeuninit stuff that overlap" vs "distinct options with an active-variant rule for enum-but-without-stored-discriminant". But those are definitely not this RFC.)

[RustyYato]

I don't mind spelling it as repr(offset_zero) for unions if that helps get this RFC accepted 😄. However, I have a sneaking suspicion that this isn't the contentious part of this RFC.
I know the name isn't optimal (intentionally). This can be hashed out after the RFC is accepted (or even give a different name for all of struct, union, and enum).
The most important bit for me is just that we do the split (for all of struct, union, and enum, to be consistent).

[RustyYato]

I've updated how enums's tags are specified, now they just defer to whatever repr(C)'s tag type is. This is done to reduce the friction of switching from repr(C) to repr(ordered_fields). To ensure that all uses of repr(ordered_fields) can be cross-platform, I've adding a lint to ensure that the user also adds an explicit repr for repr(uN)/repr(iN).

[Mark-Simulacrum]

I think this is too noisy. Most code out there using repr(C) is probably fine - IIUC, if you're not targeting Windows or AIX, maybe definitely fine? - and having a bunch of allow(...) across a bunch of projects seems unfortunate.

Maybe we can either (a) only enable the lint for migration, i.e., the next edition's cargo fix would add allows for you or (b) we find some new name... C2 for the existing repr(C) usage to avoid allows. But (b) also seems too noisy to me.

[clarfonthey]

Maybe it could just be an optional edition compatibility lint, so if someone enables e.g. rust_20xx_compatibility it shows up but otherwise not.

[RustyYato]

(a) only enable the lint for migration

That was the intention, hence the name edition_2024_repr_c. I'll make this more clear, that this is intended to be a migration lint.

Rustfix would update to #[repr(ordered_fields)] to preserve the current behavior. For the FFI crates, #![allow(edition_2024_repr_c)] at the top of lib.rs would suffice. If you have a mix of FFI and non-FFI uses of repr(C), then you'll have to do the work to figure out which is which, no matter what option is chosen to update repr(C) - even adding repr(C2), since then the FFI use case would need to update all their reprs to repr(C2).

Overall, I think this scheme only significantly burdens those who have a mix of FFI and non-FFI uses of repr(C). But they were going to be burdened no matter what option was chosen.

[RustyYato]

Is the new wording/lints too noisy still?

[joshtriplett]

I think the new wording is still too noisy. We shouldn't assume that most people using repr(C) are using it for ordering rather than FFI.

[RustyYato]

That wasn't my intention, but I don't see another way to do all of the following in the next edition:

• make repr(C) mean - same layout/ABI as what the standard C compiler does
• make repr(ordered_fields) - the same algorithm that's listed for repr(C) in the Rust reference
• ensure that everyone who upgrades to the next edition gets the layout they need (as long as they read the warnings and follow the given advice)
• make it as painless as possible for people who don't mix FFI and stable ordering cases (which I suspect is the vast majority of people). In other words, each crate currently uses repr(C) either exclusively for FFI or exclusively for some stable layout.
• for people who do mix FFI and stable ordering cases in one crate, at least the warning should give them all the places they need to double-check, and they can silence the warning on a case-by-case basis.

I'm open to suggestions on how to handle the diagnostics. Within these constraints, I think my solution is the only real option we have. If there are some objections to these constraints, I would like to hear those too, maybe I missed the mark with these constraints, and missed a potential solution because of it.

[RustyYato]

@DemiMarie said this in a comment:

Would it be possible to rename repr(C) to something else? That way repr(C) becomes a compile-time error rather than silently changing behavior.

I laid out my design axioms above. I believe that making a new repr and deprecating repr(C) is too costly. Especially when the most prevalent use-case of repr(C) is for FFI. A use-case which would benefit from bug fixes that could happen after this RFC.

[RustyYato]

I've updated the RFC, mostly with minor rewordings to improve the flow. There are some significant updates since my last comment

• Added the guiding principle that I used when writing this RFC for transparency. This way I know why I made the decisions I did
• Added a number of example migration scenarios, these are all the cases I was thinking about balancing and some that came up during the lang team design meeting
• Defined how repr(ordered_fields) interacts with repr(packed)

[Jules-Bertholet]

Currently, on an enum with fields:

• repr(C) results in a repr(C) struct containing a repr(C) union (reference)
• repr(iN/uN) results in a repr(C) union of repr(C) structs (reference)

With this RFC, what should repr(iN/uN) do on the new edition? Does it still use repr(C)—and if so, which one?

---

Personally, I think this whole thing is an incredibly confusing mess, and we should fix it properly on the new edition:

• For enums without fields, or where all fields are 1-ZST that are trivial for the purpose of repr(transparent):
  • repr(iN/uN) and repr(ordered_fields, iN/uN) use the specified integer type
  • repr(C) uses the default C enum tag type
• For all enums:
  • repr(C, struct_of_union) gives you a repr(C) struct containing a repr(C) union. You can optionally specify uN/iN as the tag type
  • repr(C, union_of_structs) gives you a repr(C) union containing repr(C) structs. You can optionally specify uN/iN as the tag type
  • struct_of_union or union_of_structs can also be used with repr(ordered_fields), desugaring to repr(ordered_fields) structs and unions. You are required to specify uN/iN as the tag type in this case
  • C or ordered_fields without struct_of_union or union_of_structs is an error (except for fieldless enums as specified earlier); struct_of_union or union_of_structs without C or ordered_fields is an error; uN/iN without all of these is an error (except for fieldless enums as specified earlier)

View changes since the review

[RustyYato]

Oh, nice find, I didn't realize that repr(uN)/repr(iN) was specified in terms of repr(C).

I think that's much lower priority, so for now just specifying bare repr(int) in terms of repr(C#reprCurrent) would be fine (preserving the current behavior). Note that for structs and unions, repr(C#reprCurrent) and repr(ordered_fields) are identical, so layout wise there isn't a difference if we used repr(ordered_fields) here either. But there is a difference in that repr(C#reprCurrent) has a specified ABI (whatever Rust is doing right now), where as repr(ordered_fields) doesn't. And since repr(int) is considered FFI safe, we need to use repr(C#reprCurrent).

For users that need a real C layout, they can just use unions and structs with repr(C) on the new edition or use repr(C, int) (and adapt their code to the new layout). The only major loss is either pattern matching, or some work to adapt the to the new layout.
I think using repr(int) on an enum with data in FFI is very uncommon due to soundness concerns (overwriting the tag would easily cause UB if that enum is ever passed back to Rust). So this justifies making it a second class citizen in the FFI case.

A future RFC can handle this better if there is sufficient motivation.

[RalfJung]

Some editorial notes :)

View changes since this review

[RalfJung]

Suggested change

`repr(C)` in current editions is an alias for `repr(C#editionCurr)` and in all other editions, it matches the default C compiler for the given target triple for structs, unions, and field-less enums. Enums with fields are laid out as a struct containing a tag and payload. With the payload being a union of structs of all the variants. This is how they are currently laid out in `repr(C)`. The calling convention of `repr(C)` will also remain the same and all current editions.

`repr(C)` in current editions is an alias for `repr(C#editionCurr)` and in all future editions, it matches the default C compiler for the given target triple for structs, unions, and field-less enums. Enums with fields are laid out as a struct containing a tag and payload. With the payload being a union of structs of all the variants. This is how they are currently laid out in `repr(C)`. The calling convention of `repr(C)` will also remain the same and all current editions.

[teor2345]

The calling convention of repr(C) will also remain the same and all current editions.

Not sure if this is meant to be "remain the same in all current editions" or "remain the same in current and future editions".

[RustyYato]

I think this section is just wrong. I'll rework it, it shouldn't be saying that repr(C#editionCurr) matches a C compiler.

[RalfJung]

The last sentence here ("Since ...") isn't a full sentence grammatically, and I also don't understand what exactly you are trying to say.

[teor2345]

Grammar nitpick

Suggested change

	This is allow by default to reduce noise, since the edition lint should do the heavy lifting. This is still provided as a tool for interested users to reduce their reliance on `repr(C)` (or it's variants) when it is probably not needed. Since the largest difference between `repr(C)` and `repr(ordered_fields)` is calling convention.
	This is allow by default to reduce noise, since the edition lint should do the heavy lifting. This is still provided as a tool for interested users to reduce their reliance on `repr(C)` (or its variants) when it is probably not needed. Since the largest difference between `repr(C)` and `repr(ordered_fields)` is calling convention.

[RustyYato]

I started a thought, and it looks like I failed to finish or edited in the middle. I'll update this accordingly.

[RalfJung]

It's odd to start with a note. This should start by describing the repr, and then maybe one can add a note to that.

[RustyYato]

I understand, and normally I would put notes at the end.

However, I think keeping the overall structure consistent across sections makes this easier to read. So I would opt to either put them all at the beginning of each section or at the end of each section. But repr(ordered_fields)'s section is super long, so putting it's note at the end would be counter-productive. So, to be consistent, I kept them all at the beginning.

These notes are just for the RFC and provides context for why decisions were made. So I think it's more useful for them to come first. They are not intended to be a part of the reference. (This is how I've used notes throughout the RFC)

[RalfJung]

All I can say is that the current structure is very confusing and makes it harder to understand the RFC. I would strongly recommend to adjust it.

[RalfJung]

Suggested change

	These two layouts are *NOT* compatible, and adding `repr(ordered_fields)` to `repr(iN)`/`repr(uN)` changes the layout of the enum!
	These two layouts are *NOT* compatible, and adding `repr(ordered_fields)` to `repr(iN)`/`repr(uN)` changes the layout of the enum! This may be surprising, but it matches precedent -- adding `repr(C)` to `repr(iN)`/`repr(uN)` has a similar effect.

Is this correct?

[RustyYato]

Yes, this is correct. It's also why repr(ordered_fields) has this behavior

[DemiMarie]

What is the plan for when is depending on the exact details of C struct layout? Should one add explicit compile-time assertions in this case?

[RustyYato]

I'm not sure what you are asking. If you are asking, when is it ok to depend on the layout being compatible with the target's C compiler, then that is when you are using repr(C#editionNext) in any edition, or using repr(C) in future editions.

[DemiMarie]

I'm not sure what you are asking. If you are asking, when is it ok to depend on the layout being compatible with the target's C compiler, then that is when you are using repr(C#editionNext) in any edition, or using repr(C) in future editions.

There are cases where one needs to not only be compatible with the target C compiler, but also make assumptions about what that layout is. For instance, that is needed when a protocol is defined by a C struct. A target with a different C struct layout could not be supported.

In this case, could one use offset_of and size_of to check that the layout is what one expects?

[RalfJung]

If you want to make assumptions about the C layout you'll have to talk to the vendor responsible for that. I don't know what kind of an answer you are expecting here, Rust obviously can't make any promises.

Obviously you can use all the usual Rust ways of inspecting the layout, just like for repr(Rust).