Train schedule here:
http://as0.mta.info/mnr/schedules/sched_form.cfm
Just put in Milford and Yankees E153rd as To and From options.
You'll see a "Yankee clipper" train listed as a "through train" that leaves Milford at 12:27 and gets in at 2:02 pm. No switch needed. Arrives right at the stadium. Lots of other times available if you're willing to change trains once at Harlem 125th st station.
Trains leave the stadium to come back home at 7:23, 7:46 and 8:06 if you want the through trains (no switching) that go straight back up. Otherwise, all other trains can change out at Harlem station and go all the way back to New Haven, stopping at Milford.
Round trip cost (buy in station) should be $31.50 per ticket.
Hope that helps.
Say hi to Jarhead (Aaron) on the train if you see him. He'll be there. Just don't drink his liquid blackout, no matter how good it looks.