I would say not to worry about other people’s opinions and just to be yourself. As long as you’re doing what is pleasing Allah than thats what matters.
Im not sure whats going on with this situation and why people are acting this way. I am sorry that you are going through this. It has nothing to do with the religion because this is not the way the Ahl ul Bayt would have treated people. I will share some hadith with you that hopefully will provide some insight.
1. Imam Sadiq (as) said: "A believer's faith is not proven unless he is even worse than a donkey's corpse in the eyes of the public."
2. Imam Sadiq (as) said: "If you want to be among my believing brothers and companions, you must prepare yourselves for the people's hatred and animosity, otherwise you are not amongst my companions."
3. Murazem narrated that Imam Sadiq (as) said: "O' Murazem! Always treat the people nicely, even if they swear at us."
4. Imam Sadiq (as) said: "There has never been a Prophet or a believer, and shall never be any, until the Resurrection Day who does not have a neighbor who bothers him."
5. Imam Sadiq (as) said: "A believer always has one or more of the following troubles. Either a person with whom he lives in the same house will close the door and not let him in, or there is a neighbor who will bother him, or someone will bother him on his way to work. Even if a believer lives on the top of a mountain, God will appoint a Satan to disturb him. God will establish a companion out of his own faith for him not to fear anyone."
6. Imam Sadiq (as) said: "If a believer is at sea on just a piece of wood, God will send a Satan to bother him."
These Ahadith are from Mishkatul Anwar Fi Ghuraril Akhbar by al-Tabarsi