Skip to content

Scrap approval of test results, introduce color scale instead#1221

Open
mbuechse wants to merge 3 commits into
mainfrom
issue/1207
Open

Scrap approval of test results, introduce color scale instead#1221
mbuechse wants to merge 3 commits into
mainfrom
issue/1207

Conversation

@mbuechse

Copy link
Copy Markdown
Contributor

Show failed (some required testcase failed) as red, missing (some required testcase missing) as yellow, inconclusive (some required testcase aborted) as green with asterisk, passed (all required testcases passed) as green.

The reduces the probability of confusing people with false positives, while no longer requiring (unrealistically) labor-intensive approvals. Besides, the two views 'table_full' vs 'table' now only differ in that the former shows draft versions; similarly for 'details[_full]'.

Further work remains:

  • remove approval column from database
  • remove approval-related code from sql.py
  • remove endpoint for POSTing /results
  • simplify, consolidate, isolate business logic
  • etc.

We're doing this incrementally... for now, let's see whether this works at all.

Show failed (some required testcase failed) as red,
missing (some required testcase missing) as yellow,
inconclusive (some required testcase aborted) as green with asterisk,
passed (all required testcases passed) as green.

The reduces the probability of confusing people with false positives,
while no longer requiring (unrealistically) labor-intensive approvals.
Besides, the two views 'table_full' vs 'table' now only differ in that
the former shows draft versions; similarly for 'details[_full]'.

Further work remains:

- remove approval column from database
- remove approval-related code from sql.py
- remove endpoint for POSTing /results
- simplify, consolidate, isolate business logic
- etc.

We're doing this incrementally... for now, let's see whether this
works at all.

Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
mbuechse added 2 commits June 17, 2026 18:08
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
@mbuechse mbuechse linked an issue Jun 17, 2026 that may be closed by this pull request
@mbuechse

Copy link
Copy Markdown
Contributor Author

I took the liberty to deploy this to the compliance monitor (after I tested it locally, of course). I found a defect, that I resolved. Now it should be good. Have a look!

BTW the views use the same color scale; the only difference is that the former shows draft versions.

This is the color scale:

COLOR_MAP = {
    -1: '🛑',  # fail
    None: '🟧',  # missing
    0: '✅*',  # inconclusive
    1: '✅',  # pass
}

I think the asterisk still needs to be explained, and it collides with the asterisk for draft versions. I'm sure we will get this ironed out.

@mbuechse

Copy link
Copy Markdown
Contributor Author

So, right now, with scs2, we have three testcases that couldn't be tested properly:

Screenshot from 2026-06-17 20-49-49

(the virtual machine didn't come up in time)

Which means: we don't know whether these testcases are actually satisfied or not.

Therefore, we now get the asterisk in the summary:

Screenshot from 2026-06-17 20-49-55

@mbuechse

Copy link
Copy Markdown
Contributor Author

Note: the asterisk for "draft" can probably just be dropped, because

a) we have the convention to call the draft "next", which is quite telling already,
b) the draft versions will only be shown in the "expert" view.

The double dagger for "deprecated" can probably be removed as well, because we have a convention to not show deprecated versions.

What remains is to put the legend of symbols at the top, and presto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scrap approval of test results, introduce third state (yellow)

1 participant