IMO, Carlton is PLENTY athletic for his size.
At his size - with Carlton's dimensions from shoulder to shoulder - how much "athleticism" do you really need?
Carlton has always had an unusual gift (for a man his size) to dance around his man with great footwork. He had it at Dematha and he has it at Houston.
Kelvin Sampson is heavily utilizing him in the screen game. Sampson rightly points out that - with Josh - all he has to do is be in the area and he is so big that the players have to make great effort to go around him no matter how close they are.
Carlton is such a big target he's almost always open. The guards at Houston are having a field day getting him the ball. He doesn't have to jump to go get it or make some dynamic move to open up a passing window.
Kelvin Sampson saw him play extensively at UConn, like everyone on this board, and he liked what he saw so much that he was calling Josh as his team was preparing for the Final Four.
That man knows talent.