The other stuff is there mostly because... well, think of a long board/stick/ruler. Grab one end and twist it along the length, it's pretty easy to spin. Grab it in the middle and spin it like a propeller and it's more difficult, even though the total mass is the same. The inertia tensor makes the object act like that, instead of just like a ball.When I worked at Stainless we had our own physics engine, and to set the centre of gravity you just gave it a vector offset from its graphics origin. Not sure what else is needed or why this other stuff exists at all tbh. (Nope, not a mathematician)
If all you want to do is have the rigid body's center of mass not be at the render model origin, you'll just need to compensate for that offset when you get the transform out of the rigid body. That doesn't involve the inertia tensor at all.
And for most purposes, just computing the inertia tensor from the model's bounding box is probably sufficient.