fix: test isolation fixes for certificates and openedx-events test mixins#38347
Open
fix: test isolation fixes for certificates and openedx-events test mixins#38347
Conversation
…ubclasses Three test classes in the certificates app were calling CourseFactory() in setUp() despite extending SharedModuleStoreTestCase. Unlike ModuleStoreTestCase, SharedModuleStoreTestCase shares a single modulestore across all tests in the class and only closes MongoDB connections at tearDownClass. Calling CourseFactory() in setUp() created a new MongoDB course (and opened connections) for every test method without releasing them, causing connection accumulation across the full test run. Affected classes: - CertificateFiltersTest (test_filters.py) - CertificateInvalidationTest (test_models.py) - CertificateAllowlistTest (test_models.py) In each case the course is only read by test methods (test data such as users, enrollments and certificates is written via Django ORM and rolled back between tests), so sharing a single course across the class is correct. See: https://github.com/openedx/openedx-platform/blob/master/xmodule/modulestore/tests/django_utils.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…vents.testing openedx_events/tests/utils.py was moved to openedx_events/testing.py in openedx/openedx-events#559 so the test utilities are included in the installed package (setup.py excludes the tests/ subpackage from the wheel). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When OpenEdxEventsTestMixin was listed after a TestCase subclass (e.g. Foo(SharedModuleStoreTestCase, OpenEdxEventsTestMixin)), it landed after unittest.case.TestCase in the MRO. Since unittest.case.TestCase.setUpClass and tearDownClass do not call super(), the mixin's lifecycle methods never ran. The workaround was to manually call cls.start_events_isolation() in each class's setUpClass, but there was no corresponding tearDownClass to restore event state, causing events disabled by one test class to leak into subsequent classes in the same process. Fix by placing OpenEdxEventsTestMixin first in the base class list so it appears before unittest.case.TestCase in the MRO. This lets setUpClass and tearDownClass run automatically through the cooperative super() chain, removing the need for manual start_events_isolation() calls. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This mixin is already included via one of the other mixins on this test class so including it again was messing with the MRO for the test classes.
feanil
requested review from
a team,
farhan,
irtazaakram and
salman2013
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
These fixes address two pre-existing test isolation bugs that were exposed when rebalancing the CI test shards (#38287). They are independent of the rebalancing work and safe to merge now.
Bug 1:
SharedModuleStoreTestCasemisuse in certificate testsThree certificate test classes called
CourseFactory()insetUp()instead ofsetUpClass(). WithSharedModuleStoreTestCase, the modulestore is shared across all tests in the class and connections are only closed attearDownClass. CallingCourseFactory()insetUp()accumulated open MongoDB connections across every test method, eventually exhausting file descriptors when run in the same process as other tests.Fixed in
test_filters.pyandtest_models.py— movedCourseFactory()tosetUpClass()inCertificateFiltersTest,CertificateInvalidationTest, andCertificateAllowlistTest. Safe because tests only read the course from MongoDB; per-test data (enrollments, certificates) is in the Django ORM and is rolled back between tests.Bug 2:
OpenEdxEventsTestMixinleaving events globally disabledOpenEdxEventsTestMixin.setUpClass()calleddisable_all_events()then enabled onlyENABLED_OPENEDX_EVENTS, but had notearDownClassto restore event state. Certificate test classes withENABLED_OPENEDX_EVENTS = []left all OpenEdX events globally disabled for subsequent tests in the same process.Fixed upstream in openedx/openedx-events#559 (released as 11.1.1). This PR updates to 11.1.1 and cleans up the interim workarounds (manual
tearDownClassoverrides and duplicate mixin includes) from the affected test classes.