I can think of a few.
Politics, Not the D vs R kind but small town and internal family politics. Never fun when you have family and neighbors pissed at you because they didn't get first shot at the elk.
@Eric Albus, said it best. " hunters want a hunt, landowners want a harvest". When it comes to Cows and does that means drive or walk to the hay field and shoot one at daylight. end of hunt. I can understand wanting to make a hunt out of the harvest, but hiking the hills, passing on animals because you want a dry cow, is just taking up my time or the space of some one else that will harvest.
Many hunters are just not the quality of hunters on HT, can not be quite, can not see game, CAN NOT SHOOT. Best case they might harvest something, mostly they just spook the elk, worst case they cripple one and it end up on the neighbors. I just don't have time for this.
My biggest fear with just letting people hunt is not that gate will be left open, cows will be shot, but that the hunters will stray onto the neighbors. OnX has helped allot with this but it is still an issue.