It is Newton's first law:
In an inertial reference frame, an object either remains at rest or continues to move at a constant velocity, unless acted upon by a force.
If by some magic, the attraction of the sun stopped, the earth would leave off on a tangent.
It all started with the formation of the solar system:
Scientists believe that the solar system was formed when a cloud of gas and dust in space was disturbed, maybe by the explosion of a nearby star (called a supernova). This explosion made waves in space which squeezed the cloud of gas and dust. Squeezing made the cloud start to collapse, as gravity pulled the gas and dust together, forming a solar nebula. Just like a dancer that spins faster as she pulls in her arms, the cloud began to spin as it collapsed. Eventually, the cloud grew hotter and denser in the center, with a disk of gas and dust surrounding it that was hot in the center but cool at the edges. As the disk got thinner and thinner, particles began to stick together and form clumps. Some clumps got bigger, as particles and small clumps stuck to them, eventually forming planets or moons . Near the center of the cloud, where planets like Earth formed, only rocky material could stand the great heat
The initial forces were probably thermodynamic exchanges of scatterings . The slow domination of the collective gravitational field condensing to a sun and planets again depends on laws : conservation of angular momentum in particular.
So as implied in the comments, the real question is "why Newton's first law".
And the answer is that laws and postulates in physics are the extra axioms imposed so that the mathematical theory fits the data. It explains the tangential velocity, but the law itself was chosen so that the kinematics would be fitted and new set ups could be explained and predicted. That is what physics is about, understanding mathematically the way nature works.