The Big 12 (as a 10 school league before Texas A&M and Missouri departed) already had a negotiated contract with ESPN and Fox. Thanks to the former Big 12 commissioner, ESPN and Fox were agreeable to paying the conference the same as if it still had 12 schools.
When Texas A&M left, the conference was obligated to find a suitable replacement quickly that would keep the Big 12 from breaching that contract. BYU would have been a suitable replacement in the minds of the top brass at ESPN and Fox, but that fell through after BYU made its unreasonable requests. The media partners then approved TCU as a suitable replacement for A&M.
This scenario repeated itself when Missouri left and West Virginia was invited. WVU had to sue its way out of the Big East (with $10 million financial backing from the Big 12) in order to join the Big 12 for the 2012 season. Had that not happened, the conference would have been in serious trouble with the media partners.
So to go back to your argument, there is no way BYU was costing the conference money just because it wasn't invited. That still holds true to this day.
It's funny that BYU's coach Mendenhall had to go public with begging the Big 12 for membership.
BYU’s Mendenhall welcomes a Big 12 invite, if only it would be extended – CollegeFootballTalk