The point of the bench is to take us out of the equation as much as possible. Choking or horsing the rifle does not help and teaches bad habits when you shoot in the field. On the bench my left hand is tucked back holding the butt rest, or BEING the butt rest. 22 LR to 358 Norma Mag. Everything you need to shoot magnums is learned with a 22 LR. Bench rest shooting builds confidence in your gear. Practical shooting builds confidence in yourself.
A retired police sniper buddy of mine says he can "guide" his shots with just a subtle pressure on the grip with his trigger hand. I don't have the time or ammo budget to get that good. One way I check my cranky gear is to have HIM shoot it. The gear is usually fine.
Once I have my rifle's group centered where I think it should be for my ballistics, I leave the bench behind and shoot field positions off my pack, or trekking poles, etc.
My actual paper groups on targets matter very little to me or the animals. I limit my hunting shots to ranges I am confident I can hit a volleyball for deer and a basketball for elk.
Knowing I can consistently hit that volleyball, I then aim for a golf ball sized spot on the animal when the shot comes.
You wrote "high impacts from various ammo." Does this mean all the shots from one brand of factory group higher?
Posting a picture of you targets might help if you want to drill down. There are lots of variables in play for accuracy questions. We'd have go through those in a rather systematic approach.