First, it's important to know that the big bang wasn't an explosion of matter into empty space - it was the rapid expansion of space itself. This means that every single point in the universe appears to be at the centre. Think of the universe as an empty balloon with dots on it. Those dots represent clusters of galaxies. As the balloon inflates, every dot moves farther away from every other dot.
The space between clusters of galaxies expands, like the rest of the universe, at an accelerating rate. (Gravity keeps the clusters themselves the same size.)
Edwin Hubble first observed this phenomenon in 1929, when he noticed that the light from distant galaxies shifted to the red end of the spectrum, as though it had been stretched as it travelled through space. By measuring the wavelengths of the light, Hubble observed that galaxies were expanding away from each other at a rate proportional to their distance from one another.
In the beginning, the universe was a single point. Where was that? It was, and still is, everywhere. Scientists even have proof: Light from the big bang, in the form of cosmic radiation, fills the sky in every direction.