If you've never filled out taxes directly (either you've used software like TurboTax, or someone else does your taxes), go look at a 1040. You'll note that the complexity of the tax code has nothing to do with the tax rate. To calculate the tax you owe, you just take your taxable income and look up the amount owned in the tax table. That's it!
The hard thing about taxes is the myriad deductions and credits we've grafted onto the tax code. Some of these credits depend on your adjusted gross income. Some depend on your Modified AGI. Some depend on your taxable income. Some are refundable, some are not. Some phase out after certain income levels. In addition, we tax different forms of income (capital gains vs wages) at different rates. If we replaced lots of those credits with direct government spending, tax day would suddenly get a lot simpler.
But somehow our discourse has managed to equate "having more tax brackets" with "complexity". This has resulted in a tremendous collapse of the top marginal rates, to the point where people earning seven figures per year pay almost the same marginal rate as the upper-middle class. Kevin Drum flags a very illuminating chart on this matter , looking at effective income tax rates:
There's no reason for the tax code to flatten out once you are in the 40th percentile of American income earners (which is roughly $60,000/year). Given the general level of underfunding of the government, we could do a lot more to push the tax rate for the top 20% and top 1% somewhat higher. Putting a cap on the mortgage interest deduction, charitable giving deduction, and tacking on a 45% top marginal rate at $500,000 or $1,000,000 would be a good start.