Skip to main content
Jimmy Jansen

A Pragmatic Testing Strategy

Published: Updated:

Years of debates with developers, POs, management, and leadership have shaped my perspective. This is what I've learned about pragmatic testing in the real world, the patterns that work, the fights worth having, and the compromises that matter.

Testing is a sensitive topic for many, and that is especially true when they’re having a conversation with someone that is relentlessly advocating for the quality of work.

Over the years, I’ve had the testing discussion with developers, product owners, management, and leadership, each approaching it from their own angle. As a principal or lead, I used to dictate testing strategy. Now, as an architect, supporting in a different capacity, I guide teams toward what works for their context. I’ve seen the good, bad, and ugly of approaches, and I’ve tried to take the best from all of them.

Why write this? Because I’ve had the same conversation over and over again. Because I’ve seen enough train wrecks to know what works and what doesn’t. And because I’ve been wrong enough times to have learned something worth sharing.

This isn’t going to be another article about the testing pyramid or TDD evangelism. This is about the messy reality of testing in the real world from someone who’s been on both sides of the argument.

Before diving in, I should be clear: I’ve seen teams succeed with completely different approaches. Consistency and alignment in the team are more important than any specific approach. This post is what I’ve learned works consistently across the dozens of teams I’ve worked with. Take what resonates, leave what doesn’t.

The Mindset Challenges

Testing is not (yet another) checkmark

I’ve seen too many teams treat tests like a compliance exercise. “We have 80% code coverage!”, while their application breaks in production every other deployment.

Testing isn’t about making some metrics going green on your CI dashboard. It’s not about satisfying your tech lead’s arbitrary coverage requirements. And it’s definitely not about checking a box on your definition of done.

This team with their 80% test coverage might sound impressive. Except their tests were garbage. They tested getters and setters. They tested that their mocks returned what they mocked them to return. They had snapshot tests that simply covered every screen. The tests passed. The application was a disaster.

Real testing is about confidence. Can I deploy on Friday afternoon? Can I refactor that nasty piece of code knowing I didn’t break anything else? When that inexperienced developer makes their PR, can I trust my tests to catch their mistakes?

If my tests don’t give me that confidence, I’m just writing code to make myself feel better. It is something I’ve done over and over, and this fake confidence always bites me in the ass.

You aren’t better than your QA

The number of developers I’ve met who think QA is beneath them is insane. The number of times I’ve heard “I write tests, why do we need QA?”, usually right before their “fully tested” feature breaks in production, bothers me a lot.

Every time I thought my code was completely bulletproof, the QA engineer spent five minutes with it and found three critical bugs. How? The QA used the back button. They opened it in two tabs simultaneously. They put an emoji in the name field.

You know, things actual users do.

QA engineers think differently than devs do. They break things creatively. They use the application wrong on purpose. They have this annoying habit of testing what I built, not what I thought I built.

A hard truth: The best QA engineers I’ve worked with have prevented more disasters than most “senior” developers have ever shipped features. When developers think they’re above QA, they’re likely the exact people who need them most. The irony is painful, the developers who dismiss QA are usually the ones whose code they catch the most bugs in.

Testing is about sleeping well at night

My personal philosophy: I test so that I can sleep at night.

I’m not talking about 100% coverage or testing every possible scenario. I’m talking about testing the stuff that will wake me up at 3 AM if it breaks. The things that, when they fail, really matter. Every application has a clear golden path, which when it fails, will make customers leave, and this is where the bulk of my attention goes.

I’ve caused my fair share of outages, some are too embarrassing to write down, but for every problem I caused I was certain that the work was flawless. This was especially a problem early in my career, when I didn’t write as many tests as I do today. I simply didn’t understand what type of testing was necessary for me to feel confident in what I’m pushing to the users.

Today I’m a lot more diligent in writing tests. They give me the confidence to deploy on a Friday afternoon and feel secure that I won’t get called during the weekend.

Testing the things I won’t get right the first time

It is hard to give good examples in testing because you quickly end with the simplest and dumb ones. One that I overuse is the calc(a, b) test. This is so trivial to write that even the most junior devs can write it in their sleep without any mistakes.

Trivial tests don’t add any value. Yes, they make the testing stats go up, and yes, they make you feel productive, but at the end of the day you know these tests won’t help you sleep any better.

I focus my tests around the problems which I find hard to get right the first time. For example, I am confident that I can write a “sliding window”, but it can be a gnarly problem to get right, so I actively test it. There is a kind of gut feeling that I’ve developed for these problems, and I recognise quite early in the development process that I would need some tests.

Mindset
  • Testing is about confidence and being able to sleep, not metrics or compliance
  • I respect QA engineers, they think differently and catch what I miss
  • I test what scares me and could wake me at 3 AM
  • It is useless to test everything
  • I test what I won't get right the first time

The Uncomfortable Truths

Untested code is technical debt

“We’ll add tests later” is the “I’ll do it after dinner” of software development. We know it’s a lie, but we keep saying it anyway.

Debt accumulates for every function that should be tested but is not. Like real debt, technical debt compounds over time. That untested function becomes an untested module, which becomes an untested feature, which becomes that part of the codebase nobody wants to touch.

I’ve worked on a codebase for a huge US company that was over 10 million lines of code, and no tests. They had huge performance problems which caused daily outages, and my team had to help them resolve the issues. The first thing I did was write some tests to make sure we had something that told us we didn’t break anything. This application was a complete minefield where I would make a small change which rippled through the application due to undocumented behaviour.

If I can’t test it, I don’t understand it

This might be controversial, but I’ve found it to be true. If I’m struggling to write tests for my code, it’s not because testing is hard. It’s because my code is bad.

Can’t inject dependencies? Refactor the code to make it less tightly coupled. Can’t test without hitting the database? Split business logic from infrastructure. Need to mock fifteen things to test one function? Split the function into smaller pieces that do less.

I used to hate writing tests because they were so hard to write, but the tests weren’t the problem. My code was. Good code is testable code. If it’s hard to test, it’s probably hard to maintain, hard to understand, and hard to change. This is where testing early during development can help. It forces you to design your code in a better way.

The many excuses to not write tests

I’ve heard every excuse. “We’re moving too fast for tests”, “It’s just an MVP”, “The deadline is too tight”.

You know what takes more time than writing tests? Debugging production issues. Fixing bugs that tests could have caught. Context switching because the QA found problems. Explaining to stakeholders why the system is down, again.

The funny thing is, the hour extra that it takes you to write tests is nothing compared to the time it takes you to fix bugs after they go live. When you have good tests, you save time. At every step you can confirm that everything works as expected and nothing breaks in other places. It ensures you don’t have to rework the feature because you can confirm you did a good job.

Truths
  • Untested code is technical debt that compounds over time
  • If code is hard to test, the code design is probably bad
  • Tests save time in the long run

Understanding Different Test Types

The Car Testing Analogy

I think about testing like testing a car:

Unit tests are like testing individual components in isolation. Does the brake pedal depress properly? Does the engine start? Does the transmission shift? We test each component on its own, without the rest of the car. Fast, focused, specific.

Integration tests verify how components work together. Does the engine actually turn the wheels through the transmission? Do the brakes actually stop the wheels when you press the pedal? I’m not driving the car yet, but I’m making sure the pieces connect properly.

End-to-end tests are like taking the car for an actual test drive. Can you start it, drive to the store, park, and come back? This is the only test that tells you if the car actually works as a car.

I wouldn’t test everything by driving the car around, that would be exhausting and expensive. But I also wouldn’t sell a car without ever driving it. All three levels are necessary to ensure that the car works as expected.

Unit Tests

Unit tests test one thing in isolation. One function. One class. One piece of logic. They’re fast, focused, and deterministic. They shine for logic. Business rules. Calculations. The stuff that can go wrong in subtle ways.

I always like to have unit tests running in the background. They provide an immediate feedback loop on whether I’m moving in the right direction. This short feedback loop is what keeps me focused and productive.

You need almost no mocking in unit tests, if you do, you need to take a step back and rethink your design. Really try to isolate the functionality of what you’re testing.

Integration Tests

This is where you test that your pieces work together. I feel strongly that an integration test should never be visual and shouldn’t rely on external services.

For all my tests I try to minimise the moving pieces. So I don’t hit real databases or external APIs in these tests. Why? Because they’re yet another thing that can fail. When my test fails, my code is broken. It should not be because the test database is down or the network is slow.

Instead, I test at the boundaries. Mock the database, stub external APIs at the service boundary. Test that my services talk to each other correctly, that my business logic orchestrates properly, that my layers integrate as expected. Just like my unit tests, these tests run constantly in the background.

I run hundreds of these, and they complete in seconds. That short feedback loop is crucial. On every change, they run quickly enough that you’ll barely wait for them. When feedback is immediate, you catch mistakes while the context is fresh. You feel empowered by your tests instead of slowed down by them. They’re the backbone of my testing strategy.

I do complement these tests with some real-DB/real-service contract tests that run nightly to catch migration/ORM drift and integration issues. These extra tests verify that my assumptions about external systems are correct, without slowing me down during feature development.

E2E Tests

When you’re testing end-to-end, you’re doing so from the users’ perspective. “What happens when I click this button?”, “Can I log in?”, “Do I have access to the features I need?“. Everything within these tests should be as close to the real world as possible. It should use a replica of the database, it should use real services, running in a real browser.

During development, I do like to use these tests to automate the feature I’m developing, but quite often I don’t decide to include them in my pull request. These tests are expensive to maintain, they cost a lot of time to run, so I want only the most important tests to run in CI.

I’ve seen teams try to test everything with visual E2E tests. It doesn’t end well. The tests take forever, they tell nothing about the internals of the application, and debugging them is like solving a murder mystery where all the witnesses are lying.

When the E2E tests only test the critical paths, it makes you take them more seriously. They no longer become something you can ignore or allow to be flaky. When they fail, you know something is wrong, and fixing it is a priority.

Testing Methodologies

There are several testing methodologies. The classic testing pyramid says: lots of unit tests, some integration tests, few E2E tests. In practice, I’ve seen testing ice cream cones: few unit tests, few integration tests, tons of manual testing on top. Or the testing hourglass: lots of unit tests, no integration tests, lots of E2E tests. Some teams aim for the testing trophy: focusing heavily on integration tests with real services, which can work but often leads to slow, coupled test suites.

I don’t really focus on any methodology. I care about writing good tests, whichever test makes the most sense for the topic I’m working on. When I work on business logic, algorithms, and calculation, I write unit tests. Whenever I integrate pieces of systems, I write integration tests with mocked boundaries. During development, I use E2E tests to make my development easier, but I don’t commit a lot of E2E tests because I reserve those for critical paths only. It’s not about following any method religiously, it’s about having the right test at the right level and keeping the tests fast and reliable enough that I can run them non-stop.

Test types
  • Unit tests for business logic and calculations
  • Integration tests for integrating systems
  • Integration tests get mock at boundaries
  • E2E tests only for critical user paths
  • Keep tests fast and reliable or they won't get run

Where I Stand Today

After all these years, all these arguments, all these successes and failures, here’s where I’ve landed:

Testing is a tool, not a religion. Like any tool, it can be used well or poorly. The goal isn’t to test everything. It’s to test what matters. The metric isn’t coverage, it’s confidence.

I’ve learned to meet teams where they are. No tests? Let’s start with one. Flaky test suite? Let’s fix the worst offenders. Too many tests? Let’s identify which ones actually provide value.

I’ve learned that the best testing strategy is the one that actually gets followed. A pragmatic approach that the team believes in beats a perfect approach that they ignore.

Most importantly, I’ve learned that testing isn’t about being right. It’s about shipping better software, sleeping better at night, and making our future selves hate our present selves a little less.

So I test what scares me. I skip what doesn’t matter. I’m pragmatic, not dogmatic. And I remember: the best test is the one that catches a bug before my users do.

That’s my testing philosophy, forged through years of successes and failures. It might not be your philosophy, and that’s fine, as long as your team is aligned on whatever approach you take. But whatever you do, please stop testing your mocks.