When I was in graduate school, I remember one stark example of the limitations of standardized tests.
There was a test, I believe long since abandoned, that was supposed to gauge children's level of "common sense".
One of the exercises went something like this - "Pretend you are playing ball in a field and you lose the ball in the grass. Draw out on this piece of graph paper how you will go about looking for the lost ball."
The test scorers were looking for something systematic - drawing rows, for example, demonstrating a methodical approach to look for the missing ball.
But scorers also noticed that a sizable minority of students drew what seemed like random lines, nothing systematic at all (or so they thought). It didn't make sense because on other exercises on the test, they "showed as much common sense" as the rest of the population.
Eventually, someone took the time to look more at who these students were, and they found a simple explanation. The kids with the random lines all turned out to live in mountainous areas of the country. The zig zag lines represented their stopping at trees on the hill, the places where rolling balls were most likely to stop.
Very common sense.