In case you haven't noticed, it's much easier for someone to put together a good basketball team than a good football team. You only need a handful of good players for hoops, but you need about 50 of them to build a good football team, even more. How many teams like Butler, or VCU, or K State rise up for a few years, and then slink back down when their hot coach gets hired somewhere else? In football, the ups and downs take longer, because you have to build up a whole roster of players, not just a few good ones.
So far, Louisville has had 2 elite coaches and you've had one. If you get a second one, then you're set. If not, well, you might soon be an Atlantic 10 team if someone doesn't come along and rescue you.