I see a lot of average players trying to pitch the ball way up in the air. Then I watch pros at tour events, and they keep the ball down, so they can better control how far it goes. Sure, sometimes you need a high shot, but your standard pitch should be a lower one that releases and rolls.
In these photos, you can see the differences between the low pitch (top) and what a lot of amateurs do (bottom). The pros play the ball in the middle to slightly back in their stance and keep the shaft angled forward through impact by shifting left on the downswing. Amateurs play the ball too far forward, then hang back on their right foot and try to scoop the ball. With this much hand action, they often wind up hitting it fat or thin—which they live in fear of doing.
Think of it this way: The pros set the loft when they sole the club at address and maintain it throughout the swing. Amateurs try to add loft at impact, which isn't necessary and leads to poor contact. So play the ball back, and keep that shaft forward. Your pitching will improve, and your fear will disappear.
It might sound obvious, but so few golfers do it: Practice your weaknesses, not your strengths. It can be hard to get yourself to spend time on something you don't like to do, but it's the quickest way to get better. Tiger was not a good wedge player when he was young, but he worked on it, and now he's one of the best in the world. Phil has always been a wild driver, but he's addressing that and driving it so much better. Take an honest look at your game, and get after those weak spots.