I have noticed that often times landowners don't even really realize that they have the right to decide. They don't want to be the bad guy and want everyone to just get along.
So whoever is the bad egg can push the issues and make things uncomfortable for everyone else and then the landowners ask the more reasonable people they have given permission to to just go along so that they don't have to confront the bad apples...
I do see both sides of the issue. My wife's family owns land that isn't really hunt able, but they do get people asking out of the blue and even though I am a hunter and they can understand why someone would ask it always makes them think that whoever is asking must have crooked intentions.
It is a very weird dynamic. Early on, I expressed interest in trying to hunt the land (this was before I saw it) and they kinda begged off the question mumbling something about how I would probably need to clear that with the farmers who rent the land for corn and beans... I tried to explain that if they didn't list hunting rights in the lease then the farmer would have no say in the manner, but they figured I was wrong. I'm not disputing that you would want to coordinate with the farmers since no one wants to be on a tractor with a firearm pointing at them from the hedgerow, but the landowner or their designee should not need permission to access the land as long as it doesn't interfere with the lessee's operations.
I think a lot of landowners are very absent from their land and they have no concept of who is or is not accessing their land. I have been involved in some hunting situations where the permission was granted to someone else and I was invited along and I wasn't always sure that the landowner was made aware. I try to avoid those situations and I've been shifting to more public land hunting. Some day I hope to have some of my own land to hunt on, but it might be a few more years before that happens.
And of course, I then worry that I will buy a place that has bad neighbors who run roughshod over all of the parcels nearby. I'll have to be diligent to avoid that.