With much to offer in support of your overall point, I additionally offer that, "Let's call a spade a spade" has an evolved meaning that has led me to feel it's best left unused.
Over many decades, I've heard questionable comments about Jews that have been waved away as innocent, ignorant, harmless, and such. I don't doubt that's sometimes true, but it's usually pretty low cost to let go of something that might be perceived or felt as degrading, harmful, threatening, or endangering when used by certain others.
Something to consider...
Is It Racist To 'Call A Spade A Spade'? : Code Switch
Is It Racist To 'Call A Spade A Spade'?