I used to work with Chris, Clay Kallam , and Glenn Nelson scouting recruits for Scout. We talked about the criteria for ratings and decided we would rank based on potential rather than the product that we currently saw on the court. FWIW, I argued that it was meaningless to have ranking that included all five positions, preferring one for guards and one for forward/centers, but that’s not what people want to see so I lost that argument.
A scout’s personal preferences come into play in rankings. I was never won over by chuckers that would score 40 points while forcing shots and ignoring her teammates, no matter how talented. I’d take a Kelly Faris over a Kelly Mazzante every day and twice on Sunday. We had some interesting, and friendly, arguments and some of the rankings were compromises to accommodate everyone’s opinions. I actually think rankings composed by a group is better than rankings done by an individual as it helps to eliminate some biases.