Skip to content

feat: add TESTS filter#1387

Open
Danyal-Faheem wants to merge 14 commits into
overhangio:releasefrom
Danyal-Faheem:danyal/add-tests-filter
Open

feat: add TESTS filter#1387
Danyal-Faheem wants to merge 14 commits into
overhangio:releasefrom
Danyal-Faheem:danyal/add-tests-filter

Conversation

@Danyal-Faheem
Copy link
Copy Markdown
Contributor

@Danyal-Faheem Danyal-Faheem commented May 11, 2026

fixes #1386.

We add the following changes:

  1. Add a new TESTS Filter that will accumulate tests across tutor core and plugins.
  2. Add a new tutor local do tests [suite] --limit=<service> command to run tests across core and plugins. For example, to run smoke tests across plugins: tutor local do tests smoke
  3. Adds smoke tests for tutor core that include (moved to a new PR):
    i. User Management and Authentication
    ii. Course import, course creation
    iii. User enrollment
    iv. Basic healthchecks
    v. OAuth Application creation

@Danyal-Faheem Danyal-Faheem marked this pull request as draft May 11, 2026 20:24
@Danyal-Faheem Danyal-Faheem changed the title [WIP] feat: add TESTS filter feat: add TESTS filter May 13, 2026
@Danyal-Faheem Danyal-Faheem marked this pull request as ready for review May 13, 2026 19:33
@ahmed-arb ahmed-arb moved this from Pending Triage to In review in Tutor project management May 18, 2026
@ahmed-arb ahmed-arb requested review from Faraz32123 and ahmed-arb May 19, 2026 09:09
Comment thread tutor/commands/jobs.py
Comment on lines +635 to +636
yield ("lms", tests_teardown_lms_template(merged["SMOKE_USERNAME"]))
yield ("cms", tests_teardown_cms_template(merged["SMOKE_COURSE_ID"]))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As TEST_DEFAULTS doesn't have SMOKE_USERNAME & SMOKE_COURSE_ID, can we make sure cleanup job doesn't fail with key error? Or I missed something?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was testing this with some smoke tests included and I removed them for the PR. I think I will add them to the PR itself which would make it easier to understand.

Danyal-Faheem and others added 6 commits May 22, 2026 19:43
…MS ports

- Add tutor/tests/smoke/ with test_lms, test_auth, test_users,
  test_enrollment, test_courses, conftest supporting LMS_PORT/CMS_PORT
- Register smoke tests via TESTS filter in tutor/plugins/openedx.py
  using (service, suite, path) 3-tuples under lms/cms contexts
- Fix SMOKE_TESTS_USERNAME/COURSE_ID → SMOKE_USERNAME/SMOKE_COURSE_ID
  key mismatch in TEST_DEFAULTS
- Auto-inject LMS_PORT=8000/CMS_PORT=8001 in dev mode so tests reach
  the dev server directly without needing manual env var overrides
- Add built-in smoke suite docs to platformtests.rst and run.rst

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

Feature Request: Add a pluggable format to include smoke/integration/e2e tests for tutor and plugins

4 participants