Skip to main content

In the #curl project, we spend 3.3 days/day on running tests - around 140,000 tests per commit/PR. In addition to what every developer runs in their own systems of course.

Our test failure rate in CI jobs is at 0.004%, which is annoyingly high when running this many tests.

Data from Dan Fandrich's curl up 2024 talk: youtube.com/watch?v=TxNdAm845T…


Test Clutch by Dan Fandrich - curl up 2024

Dan talks about his test results analytics system and the combat and struggle against flaky tests and failing CI jobs.
YouTube
#curl
in reply to daniel:// stenberg://

Lars Wirzenius
Lars Wirzenius
• •
Out of curiosity, do you run the same tests repeatedly? I do that to find flaky tests. But I don't run tests as much as you per commit.
in reply to daniel:// stenberg://

Martin Peck
Martin Peck
• •
That's a great presentation. Lots of interesting stats and insights. The 3.3 days/day stat is incredible.
⇧