Very interesting. I'm wondering whether the use of sanitizers (AddressSanitizer, ThreadSanitizer, maybe UBSanitizer) could replace some of these wrappers. They have a small overhead/footprint and are part of gcc and clang.

This is one of those when you read it you have a facepalm reaction and go about implementing in your long-running test runs... Should of course include an option to force run some tests via commit message tags or similar.

@excds often when one of these tests show a problem I rerun the tests in their entirety locally when working on the fix. The "fixed seed" per month then makes sure that the next CI round runs the same random set and thus verifies the fix properly when merged.

The fixed seed certainly verifies failures found in a test run, but if you add a new test to additionally verify something, it's probably nice to be able to run that newly added test.

@excds yeah, I suppose it would. I just never got around to add any such wizardry!

Hmm, what about always running tests that were written the last month? Then you would get the benefit of always running the newest tests together with the random ones for different code paths.

@excds I like it! That sounds like a rather simple and yet effective approach, as the newest ones are probably more interesting to run "complete"