Do you have a consistent process for aligning to the ball?
If not, it becomes a guessing game. Guessing leads to doubt, and doubt leads to missed putts. Instead, always pick a blade of grass or some spot to set your putter’s face that is in line with your putt. This promotes consistent alignment at address.
Try the Set-it-and-Forget-it Method: Align the line on your ball with your intended line. Then set the line on your putter with the line on your ball. From there, the only thing you need to think about is speed.