The problem was originally reported in https://github.com/bitcoin/bitcoin/issues/22450#issuecomment-880629375. This is a dedicated issue to track it independently because it is a different problem than the one reported in #22450 OP/description.

```
fuzz: addrman.cpp:149: void CAddrMan::SwapRandom(unsigned int, unsigned int): Assertion `nRndPos1 < vRandom.size() && nRndPos2 < vRandom.size()' failed.
```

To reproduce:
```sh
FUZZ=addrman_deserialize ./src/test/fuzz/fuzz /tmp/crash-ef7d4a4f940b8fdb4999f6a92969260cf8e1405f.log
```
[crash-ef7d4a4f940b8fdb4999f6a92969260cf8e1405f.log](https://github.com/bitcoin/bitcoin/files/6830072/crash-ef7d4a4f940b8fdb4999f6a92969260cf8e1405f.log)

*    * Addresses of nodes that are known to be accessible go into 256 "tried" buckets.
 *      * Each address range selects at random 8 of these buckets.
 *      * The actual bucket is chosen from one of these, based on the full address.
 *      * When adding a new good address to a full bucket, a randomly chosen entry (with a bias favoring less recently
 *        tried ones) is evicted from it, back to the "new" buckets.
 *    * Bucket selection is based on cryptographic hashing, using a randomly-generated 256-bit key, which should not
 *      be observable by adversaries.
 *    * Several indexes are kept for high performance. Defining DEBUG_ADDRMAN will introduce frequent (and expensive)
 *      consistency checks for the entire data structure.
 */
#define ADDRMAN_TRIED_BUCKET_COUNT_LOG2 8
#define ADDRMAN_NEW_BUCKET_COUNT_LOG2 10
#define ADDRMAN_BUCKET_SIZE_LOG2 6
#define ADDRMAN_TRIED_BUCKETS_PER_GROUP 8
#define ADDRMAN_NEW_BUCKETS_PER_SOURCE_GROUP 64
#define ADDRMAN_NEW_BUCKETS_PER_ADDRESS 8
#define ADDRMAN_HORIZON_DAYS 30
#define ADDRMAN_RETRIES 3
#define ADDRMAN_MAX_FAILURES 10
#define ADDRMAN_MIN_FAIL_DAYS 7
#define ADDRMAN_REPLACEMENT_HOURS 4
#define ADDRMAN_TRIED_BUCKET_COUNT (1 << ADDRMAN_TRIED_BUCKET_COUNT_LOG2)
#define ADDRMAN_NEW_BUCKET_COUNT (1 << ADDRMAN_NEW_BUCKET_COUNT_LOG2)
#define ADDRMAN_BUCKET_SIZE (1 << ADDRMAN_BUCKET_SIZE_LOG2)
#define ADDRMAN_SET_TRIED_COLLISION_SIZE 10
static const int64_t ADDRMAN_TEST_WINDOW = 40*60; 
/**
 * Stochastical (IP) address manager
 */
class CAddrMan