Why didn't tests save me?

Yesterday, I shared a bug I ran into with duplicate event listeners firing after upgrading to Laravel 11.

As much as I talk about the importance of testing, maybe you're wondering: Why didn't your tests save you?

Inside our feature tests, we fake events and have the listener logic tested separately.

If you look at that article, you'll see we even assert how many times an event was fired. But in this case, the event is only fired once. The problem is that the listener is registered twice.

So then I wondered if this means I should rethink our approach. If I go back to testing the listener logic inside my feature tests, it would have caught this. But I've been down that path before, and it ends up making your feature tests bloated and harder to follow.

Then I wondered if I could write some sort of additional assertion in my listener test to see if it is wired up exactly once. While technically possible, this looks really weird, and it feels like now I'm testing the framework, which I don't recommend.

I bounced these ideas around with Aaron, and he recommended we not change anything. The reality is it's not productive to try to create a coding rule or test coverage for every possible weird situation that could occur. Sometimes "things just happen."

This doesn't mean we give up and not try to test our apps at all. It just means we don't overreact to every single bug with some new technique to try to prevent it in the future.

Here to help,

Joel

P.S. Do you need an "Aaron" in your life? Someone to bounce ideas around with and make sure you're thinking clearly? Then I think you'd like the Mastering Laravel community.