Yes, you can say "sweetie" to a man, as it's generally considered a term of endearment.
The reference material indicates that "'Sweetie' is vanilla, it's steady, a surefire hit; it's the Tom Hanks of boyfriend nicknames." This suggests that "sweetie" is a commonly used and generally well-received term of affection.
However, as with any term of endearment, its appropriateness depends on the context and your relationship with the person you're addressing. Some men may find it endearing, while others may not.