Thanks to the OP for asking. I'm taking notes.
As noted, 3 days are not enough, but the crowding and expense may get to you by then.
Some more thoughts, many listed by others.
Yankee Stadium, where you'll be, and Madison Square Garden.
Ground Zero/WTC/Freedom Tower/1 World Trade. Include the little church nearby that survived.
Any theatre near Broadway. The Apollo.
30 Rock on the Today Show.
If it's at all warm, the Circle Line boat ride around Manhattan.
The double-decker bus tours of upper or lower Manhattan give you a great overview.
Park Avenue, Madison Avenue, Fifth Avenue, Central Park South, Central Park West.
Pick a few cuisines and sample them. Regarding pizza, remember that it's NYC and not Chicago--thin crust.
There's a rice pudding place in Lower Manhattan.
If you go to Little Italy, 2 cardinal rules: sit facing the door, and frisk anyone who goes to the rest room in the middle of dinner.
The post office across from MSG. Soon to be the Moynihan train station. The informal USPS motto is engraved in granite on the facade.
If you like architecture: Flatiron, Seagrams, Citi, IBM, etc.
Note the new, narrow 1000' tall condos in Midtown priced at up to $100M.
Grand Central and Times Square.
Central Park, Bryant Park, Battery Park.
St. John the Divine, Riverside Cathedral.
The U.N.
The quad and library at Columbia. NYU and Washington Square.
Macy's, Bloomingdales, Saks, Lord and Taylor.
Tiffany, Gucci, Fendi, LV, et al.
Museums: several great art galleries; radio and television; the City of NY; tenement; and sex (blush).
The Plaza, Essex House, Four Seasons, Waldorf-Astoria, Pierre.
Have FUN!