Today I heard an argument to prove that the Earth-Centered Earth-Fixed (ECEF) reference frame is non-inertial. It seriously doesn't make any sense to me but I also heard that same argument was made by Einstein, so I am asking it here. Please explain how it proves that Earth frame is non-inertial. The argument goes like this: Suppose the Earth frame is inertial. Now, all the heavenly bodies will have to go around the Earth once a day. But, this implies a very fast speed. Hence, the Earth frame can't be inertial.
Your argument is actually more or less right, but some of the details are wrong.
First you have to realize that Newtonian mechanics and general relativity have different definitions of an inertial frame. According to Newtonian mechanics, the coffee cup sitting on my desk right now defines a (very nearly) inertial frame, but a falling rock is extremely noninertial, because the rock has an acceleration of 9.8 m/s2. According to GR, free-fall is the preferred inertial state, so the rock is considered to define an inertial frame, but the coffee cup has a proper acceleration of 9.8 m/s2.
The Newtonian definition is actually impossible to define 100% rigorously, but traditionally the "fixed stars" have been taken as a pretty good standard for Newtonian frames. Any frame in which the stars have a very small acceleration is considered a very good inertial frame.
So if you have the Newtonian definition in mind, then your argument only goes wrong at the end, where you refer to a "very fast speed." What's relevant is the stars' acceleration, not their speed. If a rocket ship is gliding through our solar system at 1,000,000 m/s, then it's an inertial frame. It doesn't matter that the stars have a velocity of -1,000,000 m/s in its frame; what matters is that they have a=0.
According to the Newtonian definition, a frame of reference fixed to a point on the earth's surface is not an inertial frame. You can tell this because in that frame, the stars have large centripetal accelerations. However, the earth-fixed frame comes very close to being inertial, because you can find other frames that are inertial and that differ from it only by a very small acceleration. Therefore experiments on the earth's surface need to be pretty sensitive in order to detect any noninertial effects. The classic example of such an experiment is the Foucault pendulum.
In GR, a frame of reference fixed to a point on the earth's surface is not an inertial frame, and it doesn't even come close to being one. It differs from a valid (free-falling) inertial frame by a huge amount -- an acceleration of 9.8 m/s2. Even an extremely crude experiment can determine this. For instance, I can tell because I feel pressure from my chair on the seat of my pants. A secondary issue is that the earth's frame is rotating, and GR does consider rotating frames to be noninertial as well. (There was a lot of historical confusion on this point, including some early mistakes by Einstein, who thought GR would embody Mach's principle better than it actually did.)
In a frame of reference attached to the surface of the planet, everything far away (other planets, stars, distant galaxies...) follows a circular (or nearly) path with a period of 24 hours. These paths pose two problems
Condition one runs afoul of Newton and condition 2 of Einstein.
On the other hand, a non-rotating frame attached to the Earth or a non-rotating frame fixed to the CoM of the solar system both have no trouble with the rotation and are OK in general relativity (because they are freely falling).
You can of course, see any upper-division or graduate mechanics text for a discussion of transformation to and from non-inertial frames that allow you to overcome all of this.