I think this is a very valid question. The nature of my job before retirement was to also look at the potential corner cases/worse-case scenarios to see how brittle an "architecture" was. In this case, I would hypothesize the following:
NR fees continue to rise to the point that only NR hunters with wealth can afford to have access to a public resource outside of the state residents. As it applies to federal lands, that leaves a large population of average NRs paying for management of a public resource (via federal taxes) to which they have effectively no access to use. It becomes very easy then for that segment to demand that wildlife resources on Federal lands are subject to Federal hunting management and hunting permits/licenses become a nationalized affair vs a state affair. They can then argue that Federal tags are equal cost whether resident or NR to fairly share the management cost burden. Given the far larger population of those states east of the Mississippi, and the associated political representation that goes with it, western states may find themselves subject to a "democratic" decision they really don't want. There is a parallel discussion around landowners monetizing a public resource via landowner preference programs, hunting access programs tied to set-aside tags, etc. But in effect, States are doing the same thing relative to NR hunters with a resource that should be available to all of the public.
Is this scenario at the extreme? For now, yes (possibly). But for how long? When a public resource is only really accessible by the very wealthy, it is far easier to argue that if only the rich can do it, it's time to take that away from everyone.
Honest debate and criticisms always welcome.