Not at all.
I will say make sure you put on your hunting ambassadors hat and put an extra scoop of patience in your coffee.
Last year I hunted above a super popular hiking trail in Aspen and packed a buck down at 11am on a Saturday morning. Parking lot was at capacity and the trail was packed. I was stopped numerous times by folks. A couple of people, I think the anti's, just steered clear, but a lot of people showed genuine interest in what I was doing.
The non-hunting community seems to have this idea in their heads that hunters are all running roads, drinking bud heavy while crushing marlboros, and shooting at everything that moves.
If you deviate from this paradigm they are surprised and interested.
Be friendly/ polite, wear a mask when appropriate and you won't have a problem.