Mind · Reasoning
SelfCheckingBias
Correcting as you go still fails when the same model that produced the shape is the one checking it
Definition
Mechanism
Updating a model with feedback on its current output, as a drone re-plots its course or a team reviews its own plan, improves on a frozen decision. But if the check uses the same perspective and the same information as created the shape, systematic errors pass unseen: a drone flying confidently to the wrong address keeps correcting its course toward the wrong target. Fresh data on position does not reveal a wrong goal. This is a remaining form of MonoModularity inside a corrective loop. It is overcome by checking against a complementary, overlapping model, as staff who know particular rights-holders can spot a payment error the database cannot, which is the contribution of MetaModularity to ShapeControl.
Invariants
- A correction loop that uses only the original model's perspective cannot detect that model's systematic errors
- Detecting systematic error requires an overlapping, non-identical model
- Feedback alone does not distinguish correcting toward the right target from correcting toward the wrong one
Risks
Failure modes
Treating frequent feedback as proof of meta-modularity
Reviewing a plan only with the people and data that produced it
Mistaking a hierarchy of data about the model for an independent model of the domain
Relationships
Builds on
References
A body plan that holds many overlapping but non-identical models of the same problem, ready to instantiate whichever fits
A body plan in which one model processes every situation, treating each situation as just more data for the same process
Keeping an organisation, model or trajectory close to a desired form or target, with information serving that form
Used by
References
When the total investment is not known in advance, add corrective information as mass is added, from more than one model