Re: why have ANY upper limit?
Well, properly done, the password isn't stored. Only the hash of the password is stored. So, picking a password is just equivalent to picking one random number a the hashed password value.
For instance, suppose the hash of the password is a 32 bit integer, and the password can be a 36 bit integer. Then in fact the password that I actually use id not the only password input that generates that hash. There are 15 others, and inputting any of the 16 will be accepted as if it was my password.
Now, a thoughtful attacker still will try out "password" and "123456" first, because those might be the actual password.
32 bits is 4 bytes or, roughly speaking, 4 ASCII characters, but if that was also the password length then "abcd" is still likelier for someone to use, unwisely, than "J£@2". However, the set of all 8 character hexadecimal expressions will map into all 4 byte values if thr hadh works well. So for instance "61626364" ... well, that's "abcd" again, but better.
If you type octal triplets then a password format of 012 345 670 123, with random numerals 0 to 7, may be memorable one triplet at a time, and convenient to type, and it has 2^36 combinations, so it includes basically all of the 32-bit hash values. So adding more digits than that to this password does not increase protection.
But of course it's a false assumption that a password hash is 32 bits... we hope. Today, it needs to be substantially longer than that. And so a password length and format that exhausts the possible hash values is much longer too. But such a limit still exists, so your password may be the entire second act of Hamlet but its hash happens to be the same as "abcd", so I can just type that and crack your account. What are the chances eh? True - extremely unlikely, but there will be much shorter duplicates. And the curse of long passwords, mistyping when you want to log in, is worse for longer.