It is not blasphemy because you are basically God to damn someone or something. But generally it’s not appropriate because people use it to curse others. Yes one can say it on evil people and enemies of God. For instance one can say “Oh God, damn the evildoers or damn Satan.” That would be fine