Just close your eyes and
ask yourself:
-What does it mean gossiping for me?
-What I’m afraid of?
-Why is bad to be alone? Alone? Or lonely?
-Why I would rather be with such people in uncomfortable situations ?
Feel your body. Body will give you answers.

Now, because you are having free will to choose and make decisions. Who holding you with people who are getting annoyed of true you???

Remember you are the one who decided to be in such situations with such people.

Even Islam teaches us to choose companions.
“It is important that we surround ourselves with righteous friendships, the kind that provides goodness, security, and remembrance of Allah. The Quran encourages us to cultivate such relationships. Allah the Almighty Says (what means): {And keep your soul content with those who call on their Lord morning and evening, seeking His face, and let not your eyes pass beyond them, seeking the pomp and glitter of this Life; nor obey any whose heart We have permitted to neglect the remembrance of Us, one who follows his own desires, whose case has gone beyond all bounds} [Quran 18:28]”