1) Mystic Aquarium - It’s an aquarium. In Mystic.
2) Maritime Aquarium in Norwalk - It’s a maritime aquarium. In Norwalk. The Hotel Zero Degrees is nearby and highly recommended.
3) LEGOLAND in Yonkers - I wouldn’t go, but there it is.
You have about three different ways to get back to Boston from Hershey - two will take you through the NYC or Westchester area and one will take you through New York on Route 84. They will all take you the same amount of time, so this is not really a detour....
4) Rocking Horse Ranch - Google it. Great place for kids in Highland, NY.
5) Thayer Hotel at West Point - Hotel is on campus at West Point Military Academy. Woodbury Commons, heap-big shopping place is nearby.
6) Anywhere in Poughkeepsie/Highland/New Paltz/Kingston/Rhinebeck - See the Walkway Over the Hudson, take a cruise on the Hudson, etc. If you want to stick right near 84, there are a ton of hotels in Fishkill and there’s a good-sized water park nearby.
7) Mid-Hudson Valley - You and your kids can sit on my deck and look in the sliding glass window as I craft artisanal postings for the Boneyard.
These will take you a bit farther north, but not near Albany.
8) Google Opus40. You can stay nearby Saugerties...
9) Saugerties Light House - you can stay at an actual lighthouse on the Hudson River. There’s a tiny bit of roughing it involved, I believe, but not much.
There are a bunch of other places to stay, with a four and a six year old, it’s a little limiting. They’d likely be happy to hit something like the Pearl River Hilton and just goof off in the pool.