Riders on the storm, into this house we’re born
Bayesian probability is a great model of rationality that gets lots of important things right, but there are two ways in which its simple version, the one that comes most easily to mind, is extremely misleading.
One way is that it is too easy to assume that all our thoughts are conscious – in fact we are aware of only a tiny fraction of what goes on in our minds, perhaps only one part in a thousand. We have to deal with not only “running on error-prone hardware”, but worse, relying on purposely misleading inputs. Our subconscious often makes coordinated efforts to mislead us on particular topics. (…)
We may see one part in a thousand of our minds, but that fraction pales by comparison to the fact that we are each only one part in seven billion of living humanity.
Taking this fact seriously requires even bigger changes to how we think about rationality. OK, we don’t need to consider it for topics that only we can influence. But for most interesting important topics, it matters far more what the entire world does than what we personally do.
