 Why don't the cousins realize the flaw in their plan?
And, on those worlds, it's far cheaper to use native labor than to ship out all that expensive machinery. Conrad's idea had been to go back in time and "eliminate!" In that way, there would be no space-drive, and, hence, the Terrestrials would never get to the other planets and oppress the local aborigines. "I wormed the whole thing out of him and all of us—the other cousins and me—held a council of war, as it were, and we decided it was our moral duty to go back in time ourselves and protect you." If one of us guarded you night and day, he would never be able to carry out his plot.
 They are not very intelligent.
 They do not understand time travel.  They all originated from the same point in time.  They are highly interbred.  They are not very intelligent.