Boston: +1000
Freedom Trail is a must, but don't rely entirely on the guides (tours start on the Commons): some are good, some not so much, and they don't do the whole trail. You've got to cross the bridge and take in Bunker Hill on your own. So, best to buy a historical guide to supplement the sometimes silly stuff spun by guides and to complete the Trail. The Museum of Fine Arts (MFA) is a seriously excellent art museum, and the Museum of Science (just into Cambridge) is very cool and entirely kid-friendly.
North End is always crowded, but has some of the best NE seafood: the microscopic hole-in-the-wall Daily Catch is now far too well known, but definitely worth the 45 minute wait; and Neptune is legendary--and so is the inevitable 2 hr wait. For Italian pastries, it's a pleasurable battle between the two across-from-each-other giants, the Modern and Mikes, which is a win-win (except for your waistline). Some say Mikes has better cannoli and the Modern better (cream filled) lobster claws. I dunno. I'd have either at either as part of a last meal on earth....
For a fancy meal, it gets a bit more difficult. Not to sound sacrilegious, but it's easy to overspend for only a so-so meal in Boston. Our absolute fav is Craige on Main in Cambridge. Expensive, very informal, and always delicious. Generally the better value in Boston is in the less expensive food, I think.
If you stay in Hartford, the newly-renovated Athenaeum is a must. A great, great regional museum and a fantastic place to eat lunch. There are other things to do there, too, but, frankly, it'd be hard to fill-up four days in Hartford, and Boston (and Cambridge) is so much the greater attraction. If you are in the Boston area, consider spending a day in Concord. Be a serious historical tourist at the spot where was fired "the shot heard round the world." Been there a dozen times and still get goosebumps. Sleepy Hollow Cemetery (Author's Ridge) is hallowed ground for American literature and thought, and of course Emerson's house (pretty good tour, but the house closes in mid-Oct) and inevitably Walden Pond. Bring a copy of
Walden and read from it on the spot of Thoreau's house. There are a zillion iconic things to do in Concord (no offense to Lexington!). And from Boston, it's an easy drive to Salem, MA, which is great fun (house of witches, etc), and not far from the terrific Peabody-Essex museum. And when in Essex, of course, do a pilgrimage to Woodman's for the most famous fried claims in the world.
Can your family stay for four months instead?