Would it be completely unreasonable to allow `fundrawtransaction` to use unsafe outputs?

I understand this means the resulting transaction may be invalidated whenever the unsafe output I'm relying one disappears (e.g. because the transaction that produced it was RBF-ed) but in my case that's ok, I'll react to that and re-publish the tree of child transactions accordingly.

But let me know if you think that's too much of a footgun for users, in that case I'll architect my solution differently.
for (const auto& txout : wtx.tx->vout)
            if (IsMine(txout))
            {
                CTxDestination address;
                if(!ExtractDestination(txout.scriptPubKey, address))
                    continue;
                grouping.insert(address);
                groupings.insert(grouping);
                grouping.clear();
            }
    }
    std::set< std::set<CTxDestination>* > uniqueGroupings;  a set of pointers to groups of addresses
    std::map< CTxDestination, std::set<CTxDestination>* > setmap;   map addresses to the unique group containing it