Empiricism and rationalism in software testing

Imagine you are reviewing some software that looks like this:

do_behavior_i_want()
if get_random_int() == 42:
    delete_all_my_data()

In short, 99% percent of the time, this software will do what you want and 1% of the time it will unwantingly delete all your data. Now, let’s review this software two different ways.

Empirical review

Empirical knowledge is knowledge that comes from observation and experiment. So, let’s design an experiment to test that this software works correctly. We run the software multiple times and the software “does what we want”. Nice!

Rational review

Rational or a priori knowledge is knowledge derived from reason, deduction, and logical analysis. Alright, let’s reason through the code expression by expression. First expression, “does what we want”, great. Second expression, “does not do what we want”. Oh no! How’d that get there?

Are we talking about vibecoding?… again…

One thing I’m finding often gets missed in the vibecoding discussion is the importance of structural review. If we only do behavioral testing, we may miss “black swan events” in our software that would make it less reliable.

Empiricism can tell us what software does. Rationalism can tell us what it might do. One keeps our systems running today; the other helps us sleep better tomorrow. We need them both.