I've bought (7) farms in the last +/-35 years, several of which are primarily recreation / hunting ground with some crop ground to generate some income, pay taxes, etc.
If it's a piece you really want, then you had better act swiftly. In most cases you will only get a chance to buy that piece of ground once in your lifetime. If you lose it now, you'll probably never get another chance.
It's not going to go down in value unless it's ridiculously priced. Start with a lowball offer, you can always go up in price. So long as you don't owe a bunch on it and haven't overpaid a bunch for it, real estate can usually be sold relatively quickly and turned back into cash in the event your plans change.
Having said the above, recreation ground is a "toy" just like boats, motorcycles, airplanes, etc., except in this case it tends to increase in value over time. I don't recommend borrowing for toys and would not do it if it put other areas of my financial security at any type of risk.