Five years into the Fermi Gamma-ray Space Telescope (Fermi) mission we have learned a great deal about the γ-ray sky, yet many open questions remain, and many new puzzles have arisen. In this contribution we will consider the science drivers for a variety of topics in high-energy gamma-ray astronomy, and how these drivers map into design considerations for future gamma-ray instruments in the energy range above 5 MeV. Specifically, we take the performance parameters and data set of the Large Area Telescope on the Fermi observatory (Fermi-LAT) as a baseline, and consider the scientific questions that could be probed by improving those parameters. We will also discuss the current state of detector technologies used in space-based γ-ray telescopes and discuss the magnitude of advances that would be required to make a future Fermi-like mission transformational enough to warrant the cost and effort. These summaries are intended to be useful for selecting technologies and making basic design decisions for future γ-ray telescopes.