Wow I hadn't heard that about the hostile treatment before. The one time I went we also didn't stay at the team hotel, but Phil did get us to a party there. We had a good experience at the place where we stayed, and I thought we were treated great by the islanders.
We never felt unsafe. We felt unwanted. We stayed in a complex across the island from the games, and the people there were friendly and helpful. But we love to be off the beaten track, so we stopped at stores and restaurants --oriented to tourists but not downtown -- and people barely tolerated us. I remember stopping for directions at a store at one rural intersection and the four or five guys who were standing in front could barely bring themselves to talk with us.
And yes, I was aware of them being our fellow Americans at that time and have continued to be aware of it as we watched the tragedy that was the aftermath of the hurricane develop both in Puerto Rico, where they also are our fellow Americans, and the Virgin Islands.
If you google "Safety in the Virgin Islands," you'll see that the issue of crime has been a greater or lesser problem there over the years. Truth is, many of these people, our fellow Americans, are poor as churchmice. Other than jobs tied to tourism, mostly with no benefits and often part-time, there is little work. It is not hard to see how and why we tourists, their fellow Americans, would be resented.
I'm not saying to not go, or that you will have a bad time. I'd just advise to not do what we did. Stay where the team stays, or at a facility close by. Enjoy the beaches, which are beautiful. Watch the games, in (as someone else pointed out) a really unique atmosphere. And just be sensibly careful.