One of the one things aside from the fear of failure, that I have allowed to hold me back is the fear of growth. It sounds crazy, but the aspect of comfortability isn't always a good thing. It's more than possible for someone to be comfortable in a bad situation.
The fear of growth isn't limited to relationships, it's something that exist in your career, personal development, mindset, etc.
How many times have you found yourself unhappy at work for whatever reason, and refused to upgrade your resume because you were comfortable?
How many times have you thought about picking up a new hobby, but didn't because you weren't sure what to expect?
It's okay to grow and evolve. You could fail, but you could also soar. The only fear you (I) should have is being stagnant in life and allowing doubt to stop us from reaching our full potential. If you leave some things, habits, and people behind, be thankful. Choose your progress over anything. It's okay to grow!