Two things. Playing a physical style in the NCAA's can be a problem since there won't necessarily be Big East or Big 10 officials doing the game. Games, especially early rounds, seem to get called tighter/touchier. So depth and bench play can make a big difference. And, of course, playing tight D without fouling.
Second, there are at least two dimensions of toughness. One is physical toughness. We have guys that can be a little soft at times, but as a whole, I'd say the team has that. The other dimension is mental toughness. I feel like the team usually comes out and try to establish themselves as the Alphas on the court. When it works, the results are excellent. But, when we've met another team with comparable mental toughness, it seems like we can get pushed out of our game, and uncharacteristic play (e.g. hero-ball) starts to show up. I think that's the "toughness" that others in this thread are suggesting isn't as proven.
And that ties directly into "game smarts". That's the area that CV improved on so much last year over his JR year. He was given the ball to make something happen late - as JB was last game - and he started to find open teammates, draw fouls, give it up and get it back in better spots. I feel like that had a lot to do with the progress the team saw at the end of last year. That and Vital having JB as Robin to his Batman (or vice versa). Cole (maybe Sanogo?) needs to be JB's Robin for us to win these tight, tense games. This is where Andre's lost time to injury really hurts as he has the ability to be Robin (and eventually Batman) but needs more experience to be a more dependable creator in college.