At just 28 weeks' gestation, the babies appeared to discriminate between different syllables like "ga" and "ba" as well as male and female voices, the BBC reported.
The French team of researchers said that it was unlikely that the babies' experience outside the womb would have affected their findings.
The research lends support to the idea that babies develop language skills while still in the womb in response to their parents' voices.
Experts already know that babies are able to hear noises in the womb - the ear and the auditory part of the brain that allow this are formed by around 23 weeks' gestation.
But it is still debated whether humans are born with an innate ability to process speech or whether this is something acquired through learning after birth.
The authors of the study said that environmental factors are undoubtedly important, but based on their findings they believe linguistic processes are innate.
Dr Fabrice Wallois and colleagues said: "Our results demonstrate that the human brain, at the very onset of the establishment of a cortical circuit for auditory perception, already discriminates subtle differences in speech syllables."
But they added that this "does not challenge the fact that experience is also crucial for their fine tuning and for learning the specific properties of the native language".
Their brain scan study was carried out in the first few days following birth, so it is possible that the noises and sounds the newborns encountered in their new environment outside of the womb may have triggered rapid development. However, the researchers doubt this.
Prof Sophie Scott, an expert in speech perception at University College London, said the findings supported and added to current knowledge.
The study is published in Proceedings of the National Academy of Sciences (PNAS) journal.