All good advice above especially "If you have to think twice about shooting, you probably should pass the shot." If you feel, for even a brief moment, that you'll later dismiss the deer by saying "it wasn't what I was hoping for" then you need to suck it up, swallow hard, and let him walk. Definitely much easier said than done!
Scarlett Dew had a good point about trail cams (or scouting in general). You have to know the deer you want are in your hunting area. If not you need to move your stand to where the big guy is. This could be on another spot of your property where you tend not to look.
Tough decision choosing between empty freezer or smaller buck. You have to commit to your priority.
...and don't worry, tag soup is balanced by good experiences in the woods.
Good luck.