You are right. I attended Friday night's game and Millone absolutely dominated the Yankee lineup. Their other pitchers did much the same. They've got a young team that is totally feeding off the burgeoning confidence. They're an easy team to root for.
And riding some really streaky players right now. However, at the beginning of the year, Oakland's offense was one of the worst as a collective whole. I wouldn't be surprised if they bombed after the trade deadline (see Pirates last year).
You may be right about their offense struggling to score runs as they don't seem to have any real power guys outside of Reddick and Cespedis, but that pitching staff should continue to carry them and keep them in games.