Conference road games are tough. Baylor has some talent but they're also 14-13 overall and 4-10 in the conference. They're not exactly a great team.
I don't know about it being Baylor's SB. Baylor has already played Iowa State, Houston and Texas Tech (before Toppin got hurt) at home. So they've already hosted a bunch of highly rated teams. Plus this is only Arizona’s 2nd season in the Big 12. It’s not like this is a rivalry game. I bet Baylor and their fans would rather beat Kansas even though Arizona is ranked higher (though they don't play Kansas at home this year).