[py] reject negative enum and union index in binary decoder#3876
Conversation
|
Ooof, my apologies: it looks like these changes are all contained in #3861 submitted last week. It adds a lot of additional detailed validation around potentially invalid data, and it seems like the right PR to merge. Please don't be discouraged, there's plenty of bugs remaining to fix 😄 For what it works, this one LGTM. |
|
@RyanSkraba understood, and thankyou for informing. i will be looking forward to keep on contributing . |
|
If you're a strong python developer, we'd love some help on PR reviews (including #3861) |
|
Happy to help on reviews. Left comments on #3861 — the index checks there cover the same three sites this PR did, plus a lot more, so no complaint about closing this one. Flagged one perf regression in the collection path with numbers and a suggested fix, otherwise it looks good to me. |
What is the purpose of the change
read_enum,read_unionandskip_unionin the Python binary decoder read the enum symbol index / union branch index from the wire and validate it with an upper-bound-only check (index >= len(...)). A negative index is not caught. Because the index is then used directly to subscriptwriters_schema.symbols/writers_schema.schemas, and Python lists accept negative subscripts, a crafted negative zigzag index passes validation and wraps to an element counted from the end of the list. For an enum this silently returns the wrong symbol; for a union it selects a branch that the writer never declared and decodes the following bytes with that wrong branch schema, which is a decode-time type confusion and desyncs the stream forskip_union. Positive out-of-range indices are already rejected, so this only closes the negative side. The check becomes0 <= index < len(...)at all three sites.Verifying this change
This change added tests and can be verified as follows:
test_negative_enum_indexandtest_negative_union_indexinTestMisc, feeding a single0x01byte (zigzag-1) as the index. Both assertSchemaResolutionException. Against the unpatched decoder the enum read returnssymbols[-1]and the union read decodes viaschemas[-1], so both tests fail before the fix.test_iosuite green (152 tests).Documentation