Competency Mastery Concurrency ADR

docs/openedx_learning/decisions/0002-competency-criteria-model.rst

@@ -56,7 +56,7 @@ Decision
    - has no competency-specific constraints on associated content objects.
 
    This new database table will have the following columns:
-   
+
    1. ``taxonomy_ptr_id``: Primary key and one-to-one foreign key to ``oel_tagging_taxonomy.id``.
    2. ``taxonomy_overrides_org``: Boolean, defaults to ``false``. Used only while computing which single ``CompetencyRuleProfile`` to assign to a ``CompetencyCriterion`` (Decision 4). If, for a criterion's context, both an organization-scoped profile row and a taxonomy-scoped profile row exist as candidates, this field decides which one gets assigned: ``false`` (default) assigns the organization-scoped row; ``true`` assigns this taxonomy's own row instead, so it cannot be overridden by an organization. Once assigned, the criterion stores that one profile's id and this field plays no further part. This field is created now but read by no code path in this phase, since organization-scoped profiles don't exist yet and the conflict it resolves can't occur; see the MVP note in Decision 4.
 
@@ -240,11 +240,14 @@ Decision
    3. ``oel_tagging_objecttag(object_id)``
    4. ``CompetencyCriteria(oel_tagging_objecttag_id)``
    5. ``CompetencyCriteria(competency_criteria_group_id)``
-   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)``
-   7. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)``
-   8. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)``
-   9. ``CompetencyRuleProfile(scope_code)`` (unique -- at most one profile per distinct scope value; a plain unique constraint on the three raw nullable scope columns would not enforce this, since SQL never treats two ``NULL`` values as equal and this project's MySQL backend does not support the conditional/partial unique indexes that would otherwise route around that; see the ``scope_code`` column in Decision 3)
-   10. ``CompetencyMasteryStatuses(status)`` (unique)
+   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)`` (unique)
+   7. ``StudentCompetencyCriteriaStatusHistory(user_id, competency_criteria_id, status_id)`` (unique -- at most one HISTORY row per learner, leaf, and status level, which also serves as the idempotency key for the append in :ref:`openedx-learning-adr-0004`)
+   8. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)`` (unique)
+   9. ``StudentCompetencyCriteriaGroupStatusHistory(user_id, competency_criteria_group_id)``
+   10. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)`` (unique)
+   11. ``StudentCompetencyStatusHistory(user_id, oel_tagging_tag_id)``
+   12. ``CompetencyRuleProfile(scope_code)`` (unique -- at most one profile per distinct scope value; a plain unique constraint on the three raw nullable scope columns would not enforce this, since SQL never treats two ``NULL`` values as equal and this project's MySQL backend does not support the conditional/partial unique indexes that would otherwise route around that; see the ``scope_code`` column in Decision 3)
+   13. ``CompetencyMasteryStatuses(status)`` (unique)
 
 6. Learner progress status concepts (``StudentCompetency*Status`` database tables)
 
@@ -257,6 +260,12 @@ Decision
    - ``StudentCompetencyStatus`` tracks top-level competency demonstration state.
    - All learner status rows use a shared lookup table (``CompetencyMasteryStatuses``) so status semantics live in one place and student status tables stay structurally consistent.
 
+   Append-only history tables:
+
+   - ``StudentCompetencyCriteriaStatusHistory``
+   - ``StudentCompetencyCriteriaGroupStatusHistory``
+   - ``StudentCompetencyStatusHistory``
+
    Intended update flow (bottom-up materialization):
 
    - A learner event updates one ``StudentCompetencyCriteriaStatus`` row.
@@ -275,25 +284,25 @@ Decision
 
    2. Add new database table for ``StudentCompetencyCriteriaStatus`` with these columns:
 
-      1. ``id``: unique primary key
+      1. ``id``: unique 64-bit primary key (``BigAutoField``); see :ref:`openedx-learning-adr-0005`.
       2. ``competency_criteria_id``: Foreign key to ``CompetencyCriterion.id``
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
       5. ``created``: The timestamp at which the student's criterion status was set.
 
    3. Add a new database table for ``StudentCompetencyCriteriaGroupStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``competency_criteria_group_id``: Foreign key to ``CompetencyCriteriaGroup.id``
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
       5. ``created``: The timestamp at which the student's criteria-group status was set.
 
    4. Add a new database table for ``StudentCompetencyStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``oel_tagging_tag_id``: Foreign key pointing to Tag id
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``. This table should have a constraint to only allow status values of “Demonstrated” and “PartiallyAttempted” since it represents overall competency demonstration state, not in-progress states.
       5. ``created``: The timestamp at which the student's competency status was set.
 
@@ -422,3 +431,18 @@ Rejected Alternatives
 
       1. Silently does not work on this project's tested and production database backend. Django compiles a conditional ``UniqueConstraint`` to a partial index, which MySQL does not support; Django raises only a non-fatal system-check warning (``models.W036``) and skips creating the constraint, leaving the uniqueness rule completely unenforced at the database level.
       2. The gap would surface only as a data-integrity incident under concurrent writes, not as a test or migration failure, since SQLite (used for quick local test runs) does support partial indexes and would mask the problem in that environment.
+
+Changelog
+---------
+
+2026-07-27:
+
+* Split learner status storage into paired ACTIVE and HISTORY tables: added the append-only
+  ``StudentCompetencyCriteriaStatusHistory``, ``StudentCompetencyCriteriaGroupStatusHistory``,
+  and ``StudentCompetencyStatusHistory`` tables and their indexes alongside the in-place ACTIVE
+  tables, per :ref:`openedx-learning-adr-0005`.
+* Made the leaf HISTORY (``StudentCompetencyCriteriaStatusHistory``) index unique on ``(user_id, competency_criteria_id, status_id)``, the
+  idempotency key for the HISTORY append in :ref:`openedx-learning-adr-0004`.
+* Switched the leaf ACTIVE table (``StudentCompetencyCriteriaStatus``) to a 64-bit ``BigAutoField``
+  primary key and set ``db_constraint=False`` on the ``user_id`` foreign keys of the learner status
+  tables, per :ref:`openedx-learning-adr-0005`.

docs/openedx_learning/decisions/0003-competency-criteria-versioning.rst

@@ -44,12 +44,21 @@ For the initial implementation, versioning and traceability of competency achiev
    - A ``CompetencyRuleProfile`` is "in use" if any ``CompetencyCriterion`` assigned to it (``competency_rule_profile_id``) has an associated ``StudentCompetencyCriteriaStatus`` row. Editing an in-use profile's ``rule_type``/``rule_payload`` requires the same warning and confirmation.
    - The same warning applies when creating a more specific profile causes existing criteria to be reassigned to it, and when an authoring action switches a criterion between a profile assignment and per-criterion overrides (ADR 0002 Decision 4).
 
-5. Learner status models/tables are append-only history and do not use ``django-simple-history``:
+5. Learner status models/tables are updated in-place:
 
-   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``, each status change is stored as a new row with ``created`` as the write timestamp.
-   - Existing learner status rows are not updated in place.
-   - Current status is determined by the most recent row for a given learner + target entity (ordered by ``created``, with ``id`` as a tie-breaker).
-   - Older rows represent the learner status history and remain available for audit/tracing.
+   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``,
+     each status change updates the responsible row.
+   - Statuses only increase monotonically as described by :ref:`openedx-learning-adr-0005`;
+     downward status adjustments (for example ``Demonstrated`` to ``PartiallyAttempted``) are prohibited.
+
+6. Learner status models/tables as in 5. above each get a separate append-only history table not using ``django-simple-history``:
+
+   - For ``StudentCompetencyCriteriaStatusHistory``, ``StudentCompetencyCriteriaGroupStatusHistory``, and ``StudentCompetencyStatusHistory``,
+     each status advance is stored as a new row with ``created`` as the write timestamp.
+   - Existing learner status rows are not updated in place in the history tables.
+   - Statuses only increase monotonically as described by :ref:`openedx-learning-adr-0005`;
+     if a change would mean a downward adjustment (for example ``Demonstrated`` to ``PartiallyAttempted``)
+     or no adjustment, this does not get stored in the history tables.
 
 
 Rejected Alternatives
@@ -85,3 +94,13 @@ Rejected Alternatives
     - Cons:
         - Requires custom tooling to reconstruct past versions
         - Does not align with existing publishable versioning patterns
+
+Changelog
+---------
+
+2026-07-27:
+
+* Reworked learner status handling to match :ref:`openedx-learning-adr-0005` and
+  :ref:`openedx-learning-adr-0004`: Decision 5 now updates learner status rows in place and
+  monotonically (downward adjustments prohibited), and a new Decision 6 adds separate append-only
+  HISTORY tables. Previously a single append-only model with no in-place ACTIVE row.