Recent work indicates that there were high levels of genetic differentiation between the early farmers in the Levant, Anatolia, and western Iran. What this means that is that before the Holocene, they didn’t mix very much – probably less than one immigrant per generation. This seems to have been a general tendency in Eurasia – possibly in Africa as well, but we don’t have as much ancient DNA information out of sub-Saharan Africa yet.
On the other hand, we also see populations replacing other populations, over and over – and sometimes that resulted in admixture. Eurasians picked up a bit of Neanderthal ancestry when they expanded out of Africa. Altai Neanderthals picked up some AMH ancestry, possibly when their ancestors expanded back into the Middle East. Melanesians picked up some Denisovan ancestry.
If population A rolled over population B and occupied their territory, there was often a few percent of admixture – certainly enough to transmit key favorable alleles. But as neighbors, very little gene flow. So when people say that a little gene flow is likely to be the explanation of that vaguely Andamanese trace in Brazil, I kinda doubt it: replacement with a little admixture is known to have happened back in those days, but peaceful amalgamation had to have been rare, in order for genetic differentiation to remain high.
Of course the causes of this genetic isolationism matter. To some extent they must have been geographical barriers, particularly around the last glacial maximum. If such barriers were the only cause, then any relaxation or breach would have resulted in enough admixture to rapidly decrease genetic differentiation. On the other hand, isolation probably bred isolation: populations that had very low gene flow for long periods of time surely had very divergent languages and customs. And modern humans can effectively fission into groups that can act, sometime, almost like different species – low gene flow, even though there are no fertility barriers. Linguistic fission may go back a long time and help explain the long-term low effective population size – but we don’t know.
When people started farming, at first they were not biologically different from their immediate hunter-gatherer ancestors. Eventually they would be different: better adapted to the new diet, the increased crowding, the psychological demands of a very different way of life – but that took at least some time. After they were adapted to farming, the tendency towards farmers expanding, rather than the idea of farming being transmitted, would increase: the biological edge shows up. But at first, before much adaptation, idea transmission might have more competitive with demography. I throw this out as a possible partial explanation of the pattern we are seeing: first the idea of farming spreading through core populations in the Middle East (possibly combined with independent invention or stimulus diffusion), while a little later those core populations themselves begin spreading outward. In fact, on the ball really got rolling, those populations should have been adapting to expansion itself, just like cane toads.