Add a test suite under Tests#103
Open
TetzkatLipHoka wants to merge 1 commit into
Open
Conversation
Nine console test programs plus a script that builds and runs them with every Delphi installation it finds, for Win32 and Win64. Nothing in FastMM5.pas changes; this is purely additive. Each test exits with 0 when all of its checks passed and with the number of failures otherwise, so no test framework is needed and the runner (or a CI step) only has to look at the exit code. RunTests.ps1 exits with the number of failed runs. Covered: the block size classes including reallocations across their boundaries, debug mode basics, usage accounting per size class (both leaks and unbounded address space growth), the mode transition contract from pleriche#85, double free handling from pleriche#73, the corruption scan from pleriche#102 in both directions (detection must work, false positives must not happen), corrupted size fields in the debug header, and multithreaded stress with cross thread frees. Verified with Delphi 10 Seattle and Delphi 13.1, Win32 and Win64: 36 of 36 runs pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nine console test programs plus a script that builds and runs them with every Delphi installation it finds, for Win32 and Win64. Nothing in FastMM5.pas changes - this is purely additive, and if you never run it, it costs nothing.
The motivation is #102: the corruption scan silently stopped seeing small debug blocks, and it stayed unnoticed because there was nothing that would have run the same check across all three size classes after a change.
FastMM5Test_ScanCoverageis exactly that check, and it fails on the affected commits and passes on either side of them.How it works
No test framework. Each program is a plain console application that prints one line per check and exits with 0 when everything passed, otherwise with the number of failed checks.
RunTests.ps1builds and runs them all and exits with the number of failed runs, so it drops straight into a CI step:The
$Compilerstable at the top of the script is the only thing that needs adjusting for a different machine.What is covered
FastMM5Test_DebugModeFastMM5Test_SizeClassesFastMM5Test_UsagePerSizeClassFastMM5Test_ModeTransitionFastMM5Test_DoubleFreeFastMM5Test_ScanCoverageFastMM5Test_ScanRaceFastMM5Test_ScanHeaderBoundsUserSize/StackTraceEntryCount, i.e. the fields that decide where the scan reads; each case must give a clean report rather than an A/V.FastMM5Test_MultiThreadStressFastMM_TestUtils.pasholds the assertions and the exit code convention. It also clearsFastMM_MessageBoxEventsandFastMM_LogToFileEventsfor the duration of a run, since several tests corrupt blocks on purpose and a modal dialog would hang an unattended run.Testing
36 of 36 runs pass: Delphi 10 Seattle and Delphi 13.1, each Win32 and Win64.
Two notes
The
Tests/README.mdrecords two things that cost me time and would otherwise be rediscovered by whoever writes the next test: corrupting a freed small block is not observable (raising the report allocates the exception object, which is handed exactly that block, so the process dies before any handler runs - use medium or large blocks there), and a corruption test that happens to use a large block proves nothing about the small block path, which is how #102 stayed hidden.The sources carry
{$if CompilerVersion >= ...}guards in a few places so the same files also build on older compilers in a fork. They are inert on XE3 and later; happy to strip them if you would rather not carry them.