Motorized vehicles, electric or internal combustion, should be banned from areas that are non motorized, wouldn't you agree?
If you read any forest plan, travel plan, wilderness act, etc. you'll quickly find that non motorized areas are almost all open to foot traffic and horses, mules, etc, for all kinds of reasons already mentioned.
I also contend that the level of commitment it takes to own horses, mules, llamas, etc. very much limits access more than charging a battery on an e-bike or filling a fuel tank with hydrocarbons.