I take it you're not familiar with game theory.
ND is not stupid and they are very good at playing all sides against each other.
If ND goes to a conference, that league immediately becomes one of the strongest from an attention/tv/revenue perspective. Why exactly would any conference want to make their opponent conferences stronger by having them add ND?
And you can be sure if the others think about forcing ND to join, the Irish immediately start negotiating with all of them and suddenly the ACC schools are less likely to want ND to join a conference.
They will negotiate with the B12, and suddenly the PAC will become worried that the B12 could actually be strong enough to raid them.
Not to mention that ND will tell USC "if we join the ACC or B10, we wont have to room to play you anymore," so the Trojans also vote against it.