Try reversing the image in your heads and it might become easier to accept:
Imagine the season started with Notre Dame as the number 1 team, and UConn as the number 2. Notre Dame immediately has a freakish loss at Stanford, giving UConn the number 1 rank. UConn struggles to close wins over Duke and Syracuse, while Notre Dame goes on to destroy every ranked team they play, including Duke.
As a UConn fan you would be pretty peeved if Notre Dame got leapfrogged over an undefeated UConn simply because their wins were more convincing, would you not? The way the polls have been run thus far are pretty typical; the higher you get in the rankings, the more reluctant the voters are to have you leapfrog over other teams that also win their games. And they really dislike leapfrogging over a number 1 ranked team.