Butter is naturally yellow-toned, meaning when combined with powdered sugar, vanilla, and milk to make buttercream frosting, it ends up being off-white. Opting for a non-organic butter and giving it an extra whip are surefire ways to reduce the yellowness of your frosting, but this more effective approach may surprise you.