I'd counter that that observation is based on a skewed perspective then. Not an insult, btw. You want to see real fair weather fans, live in San Diego or somewhere like that for a while.
IMO it's really similar to the trajectory of UCONN fans I think. A lot of success breeds complacency. Success caused some of the highest ticket prices in the NFL -> less fans go to games because of it. Further, in general, with all the big screen HD TV, the viewing/camera angles, ability to stay warm and entertain yourself during commercials, it's more enjoyable to watch NFL at home. Add in that the Pats aren't the only game in (a small) town.
And to top it all off, going to a game at Gillette is a nightmare. Getting there and back is a near death experience. I'm a diehard Pats fan, but I have no interest in going to a game because of all of these things. It's a worse experience than going to a bar.