Skip to content

Contrib data handling - #2117

Draft
bfoley12 wants to merge 243 commits into
devfrom
contrib-data-handling
Draft

Contrib data handling#2117
bfoley12 wants to merge 243 commits into
devfrom
contrib-data-handling

Conversation

@bfoley12

@bfoley12 bfoley12 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reworks how Contribution.data is parsed, validated, and stored, and implements throughout CRUD paths. Also introduces pivoting a single contribution into multiple based on the presence of conditions in the data field, SI unit canonicalization, bulk-updates of Contributions (currently only used to efficiently mark many contributions as public), and string-normalization types.

Major changes:

Domain changes

  • domains/contributions/data.py: Contribution.data parses annotated keys of the form: "col_name (unit, cond1=a, cond2=b)" into properly formatted data entries.
    • "properly formatted" means we store:
      • original unit, value, error,
      • the SI converted unit, value, and error
      • precision (sig figs)
  • domains/contributions/pivot.py: expands one contribution whose data keys carry conditions into multiple contributions, keyed by condition_key
  • domains/contributions/stats.py: used to derive Project.stats and Project.columns from contributions
  • domains/_shared/units.py: SI unit normalization (Pint) and error support (uncertainties)

Model changes

  • Contribution: added condition_key as an identifier, removed needs_build, server generates _id
  • Structure.Lattice.matrix: switched from PolarsFrame to LatticeMatrix (3x3 matrix) validation to skip polars-frame encoding
  • Project: stats and columns are now derived by the server from Contributions rather than supplied, is_approved is now admin-only
  • MPContribSettings: defining controls on model relations (max contribution.data depth, max_columns, max_components, float_precision)

bfoley12 added 30 commits June 17, 2026 13:38
…rom query. Users can now request partial documents with _fields
bfoley12 added 30 commits July 10, 2026 12:05
…bution.data also updating Projects.stats and Projects.columns
…orms. Increased enforcement of Project management business logic
… parse scalar numbers to leaves, while leaving legacy Contributions untouched
…entity and sparse_always, and _all (all fields)
…hich right now only allows setting of is_public
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant