I have to disagree with Henry: I don’t think there’s been selection for ethnic nepotism. I somewhat suspect that there may have been recent selection for more accurate altruism, in some populations.
Imagine that in much of history, people lived in small groups that often fought with their neighbors. In that sort of situation, selection for group altruism is at least possible, since the group is full of close relatives, while the opponents are less closely related. Both sides are probably members of the same broad ethnic group or race, but that doesn’t matter : only the kinship coefficients matter.
Suppose that many people emerge on to the stage of history with this impulse to fight for their side: in the past, this always meant closely related people. Now, with the emergence of states, they find themselves fighting in armies, which feel like their side, but are no longer closely related – not a bunch of cousins and such. It could well be that many individuals are actually willing to risk themselves for that state. They’re willing to die for truth, justice and the Assyrian Way. It’s not genetically smart, but their adaptations are wired for past circumstances. In the same way, you might eat saccharin instead of sugar, or date a replicant instead of an actual human female , especially if the replicant looks like Sean Young in Bladerunner.
Over time, this misfiring of altruism should decrease. Patriotism burns itself out. Dying for Assyria doesn’t do your close relatives any good at all. Some people will be more prone to this, some less, and that tendency will be heritable. Those with a tendency to volunteer (in the service of anything other than close relatives) should dwindle away over time. But states are older in some places than others, and some have made greater demands than others. Imagine a region where states have been around longer, a place in which the locals have lived through empire after empire after empire. They should have had the patriotism bred clean out of them. They should feel altruistic about their families, maybe their clan – and nothing else.
Ibn Khaldun talked a lot about asabiya, a kind of social solidarity. According to him, it arose spontaneously in tribes and other small kinship groups, and could be intensified by a religion. Ibn Khaldun considered how this cohesion carried groups to power, while the gradual loss of it eventually caused the fall of the dynasty. In the conquests he describes, conquering nomadic tribes took over and started intermarrying with generic farmers and townies – all of whom had probably lost all their genetic tendencies to asabiya thousands of years earlier.
Peter Turchin has talked about asabiya’s role in the rise of empires, and how it seems to gradually be used up, never to return.
A certain amount of asabiya is probably really helpful in a republic. You need to have lots of people willing to restrain themselves when in power, and at least some willing to land on Omaha Beach. The degree of ethnic purity is not in itself key: the important things is that they haven’t had the asabiya, the genetically foolish wide-field altruism, bred out of them. If they have it, they can cooperate just as if they were related.