What do you mean with (they do not follow any religion)?
Was he Muslim, and he became Non-Muslim?
However, if you are not able to live alone away from this unhealthy surrounding, and you are forced to continue, you can tell them politely that you are Muslim, and your religion does not allow you to attend such parties.