You'd have to be a big fan of WCBB and have absolutely no fan allegiance to either institution to willingly gravitate to either of the two schools depending on which one is winning. I don't know anyone who's both a UCLA fan and a U$C fan - even less so than Cal and Stanford, which tend to have a lot of intermarriages or folks who do undergrad at one and grad school at the other.
And for what few agnostic fans there are who are simply shopping for live entertainment, the bigger problem is that the downtown LA area around U$C is saturated with high wattage entertainment of all stripes (pro sports, concerts, the arts, etc.). U$C is a very scene-y school, so WCBB is almost always too low-wattage for that campus culture. Otherwise, there are very few "locals" who could easily get to U$C games given the location, especially the Friday night games. 1.5 hours of traffic each way? Forget it.
By contrast, there are far fewer big events in Westwood and on the West side in general, and UCLA is also surrounded by much more residential housing in its surrounding neighborhoods, so it would be much less of a hassle to be a season-ticket holding UCLA fan.