No schools outside of maybe 10 nationally make a difference when it comes to academics. You're buying a commodity.
What do you mean by difference?
Many colleges have their characters and particulars. I could name a dozen ways in which the selection of the right school matters.
But just to start:
1. Rigor. There's a world of difference between many schools and others. It doesn't mean students want that, but the schools offering it attract a different kind of student.
2. Specialties. If you're targeting a discipline, whatever it is, it could be theater, film, engineering, etc., some schools have it, some schools don't.
3. Campus culture (where is a school located? Who goes there? Networks?). This is why I'd say going to NYU gives you very hard to reproduce results if you attended, for instance, Miami FLA instead.
There are a lot more variables involved.
I had a parent of my child's classmate ask me to give my opinion of a department at a New England private university that is in the middle of the rankings. Clark U. I saw that the department of interest didn't have a single permanent faculty member, they were all contingent. The kid ended up at a satellite SUNY where she'll get a much better education to prepare her for law school.