I don't think that is true in "sparsely populated areas," where there are no altitude restrictions. You do, however, need to stay 500' from people, vehicles, and structures. I don't think private land is ever mentioned. I don't know the actual Federal Aviation Regulation (FAR), but this website is consistent with what I remember.
https://pilotteacher.com/how-low-can-aircraft-legally-fly-how-the-neighbors-get-annoyed/
I should also add these regulations are about safety, not about property ownership.