I make a conscious effort to set my weight on the insides of my feet.
Setting up to putts feeling uncomfortable is an almost guaranteed way to ensure some part of you moves that shouldn't during the stroke, diminishing your chances of contacting the ball solidly and accurately.
Sometimes simply willing oneself to stay still doesn't get the job done. When that's happened to me, I've often solved the problem by making a conscious effort to set my weight on the insides of my feet or on the balls of my feet. Also, addressing the ball with the feeling that my left heel is carrying slightly more of my weight than normal has sometimes saved the situation.
I call all three of these adjustments anchoring myself over the ball, and they have rarely failed to reinforce my overall feeling of steadiness or stability at address.
START WITH THE SHAFT PERPENDICULAR
A few fine putters lay the club's shaft slightly back, with their hands behind its face at address, then return to that position at impact. It can be an effective way to get the ball rolling quickly--as long as you don't thin it or top it!
then return to that position.
Other players angle the shaft slightly forward with the hands ahead of the clubface at address and impact, either in setting up or through a slight forward press to initiate the stroke. That promotes solid contact, but because the ball tends to be driven into the green at impact, it might skid or react erratically.
I've always preferred to set the shaft perpendicular, or inclined very slightly toward the target, with my left hand directly over the ball, then try to return to that position at impact. I've found this produces the greatest consistency of roll.