…something like HMAC is used rather than a plain hash.
HMAC is a keyed hash… which means it additionally provides unforgeability.
A “plain hash” (which I assume to include cryptographic hashes) merely provides collision resistance, while a HMAC provides both collision resistance and unforgeability… because an attacker is unable to calculate a new, valid HMAC of a modified/forged message ($m$), unless that attacker also has the needed key to produce a new, valid, and verifiable HMAC.
If you would use a (let’s just call it) “unkeyed” hash, an attacker would be able to modify/change the message ($m$) if that attacker would have (for example) guessed or intercepted your encryption key, or if that attacker would be able to break your encryption in any way. The problem (better: security issue) in this case is that you would potentially be accepting and decrypting ciphertext from the attacker instead of the expected sender, and you would never learn about the fact that the message $m$ has been messed with. Your simple hash would be useless… as all it provides is collision resistance and that’s it.
When you use a HMAC, that same attacker would additionally have to successfully guess/intercept the key for your HMAC (that is, unless you use the same key for both the encryption and the HMAC… which certainly isn’t the smartest idea because that would make it easier for an attacker to forge-it-all). So, the use of a keyed hash does not only provide collision resistance like an unkeyed hash, it also lets you verify if $m$ is authentic (unforgeability).
Keeping it short and simple: creating a “simple hash” of some plaintext is a piece of cake for an attacker; but trying to create a valid and verifiable “keyed hash” is a completely different and more complicated beast. So, compared to the use of a regular hash function, HMAC practically adds an additional layer of security. That is one of the main reasons to prefer HMACs over “unkeyed” hashes.
Note that I’m not saying your
Encrypt(m | H(m)) method would be insecure (that mainly depends on the encryption… better: the used algorithm and key choice). All I’m explaining is why it’s logic to prefer a MAC over a simple hash, and that a simple hash would give you nothing if your encryption fails/breaks.
…does this not fulfill an approximation of the ideal MAC?
…no secure protocol seems to use it. Why?
If you can get an additional, well-vetted security layer for free, why would you want to avoid or ignore it?
You once said “HMAC seems a bit complicated”, but if you look at it you will notice that it also gives you a good chunk of “unforgeability” in return. See, sometimes, things are “a bit complicated” for a good reason… and people prefer to use it for exactly that reason. In this case, to gain unforgeability.