When faced with a model of the world (in physics, neuroscience, economics, ecology), how should you judge that theory? Cyrus Samii suggests 5 ways. Here is number 2:
2. If any result can be engineered then results themselves have no special ontological status.
This is another way of asking whether a model has empirical content, which we typically take as falsifiability. Yet Karl Popper suggested:
The empirical content of a statement increases with its degree of falsifiability: the more a statement forbids, the more it says about the world of experience.
And he suggested “two criteria determine the empirical content of a theory are their level of universality (Allgemeinheit) and their degree of precision (Bestimmtheit).”
I also really like the question at the start of number 4:
How complicated can the problems be that we allow our agents to solve in a model? Is a dynamic program ever admissible as a reasonable assumption on the objective function of an agent?
Recommendation – no paper on models should be published or talked about unless it makes specific, testable predictions of how the model can be tested.
I actually disagree with this rather strenuously. There are several reasons to make models, only one of which is to make predictions. Another is to confirm hypotheses.
Let’s say that you think that honeybees are dying because of the excessive use of mint toothpaste and you collect data to prove it. The problem is that data is simply a collection of facts (or “facts”) with no organizing structure. A model can give those facts that structure: you put what you know together with some of the data, and see if what you know is sufficient to replicate the observations of the world. Of course, you have to interpret these types of models carefully; they are not predictive models in the sense that they tell you anything about the world. Rather, they tell you about whether you have a consistent and complete story. But it’s still just a story.