I actually think it hurts your defense worse. A big tall 3 simply can't stay with his man in college game. Too fast, too quick, too good of handles. Those guys often become guards at the next level.
When JC tried to shoehorn in these NBA-sized lineups from 2005-2012, this is the sort of size archetype he wanted to emulate:
PG: Chris Paul (6-2); Marcus Williams, AJP, Kemba
SG: Kobe Bryant (6-6); Denham Brown, Rashad Anderson, Dyson (somewhat shorter), Lamb
SF: LeBron James (6-9); Rudy Gay, Sticks, Roscoe
PF: Tim Duncan (6-11); Josh Boone, Jeff Adrien (much shorter), Oriakhi
C: Shaq (7-1); Hilton, Thabeet, Drummond
The problem is, those NBA SF's can shoot, handle, and have the lateral quickness to defend the perimeter. In college, similarly sized guys didn't necessarily have the requisite skill. We'd often outrebound teams, but offense sometimes suffered and we'd struggle to defend certain archetypes, like Joe Alexander, Otto Porter.
A typical college lineup, and closer to what KO actually runs, is basically shifted one position "smaller". Typically 2 PGs, a 6'4-6'6 wing, a 6'8 stretch 4 type that can shoot and handle, and a big man. Those have actually been our best lineups over the last 5 years.
For this year's team, that would look like:
PG: Gilbert (Vital)
CG: Adams (Vital)
Wing: Purvis (Vital; occasionally Larrier when we go big)
Stretch-4: Larrier (Durham, Jackson; occasionally Facey or Diarra when we go big)
Big: Brimah (Enoch)
Unless guys like Facey and Diarra can play the 5, they'll only be seeing minutes on those rare occasions when we go to paint-only 2 bigs at a time, which (I hope) will be rare.