Significant differences in paternal age can make a big difference in the mutation rate. If maintained for a long time, thousands of years, this would also cause significant differences in total mutational load. For example, the average paternal age in Iceland in 1980 was 28, which would mean 56 new mutations per generation. For Australian Aborigines, which have a very unusual pattern in which fathers are about 14 years older than mothers, the average paternal age was 42.6, which would mean 85.2 mutations per generation - a 57.5% higher mutation rate. If that pattern persisted for a long time, the high-mutation population would have a much higher genetic load. Every member of that high-mutation population would have considerably more genetic errors than individuals in a population that had experienced that low Iceland rate for a long time. You would expect that this extra genetic burden would reduce brain efficiency, but you would also expect to see lots of other health problems, surely resulting in reduced lifespan.
I think that Australian aborigines may have had this pattern for a long time. The rest of the world has changed more rapidly. I know that paternal age in Iceland is higher today (33) than it was in 1980, and that it appears to have been as high or higher in the 17th and 18th centuries. But I don’t what the average pattern was over the past few thousand years.
I think this means that the differences in genetic load recently seen between European/East Asian and African populations are not particularly surprising. Particular social paths/ ecological situations can have large long-term genetic impacts, impacts that no one involved could foresee.