I’ve done many many tests over the years on multiple calibers and literally thousands of rounds down range.
Results I came to have no scientific basis. My results were based on bullet holes in paper at various distances and longevity of brass.
1: All brass is not the same. Even different lot numbers of same manufacturer of brass can vary.
2: Brass varies in wall thickness which varies its powder capacity and pressure capability and limits.
3: Brass varies in hardness and malleability - THIS is the most important variable.
4: The most effect that annealing has is it definitely will prolong brass life plus it provides more uniform neck tension on the bullet which shows up on bullet hole accuracy on paper. It will practically eliminate memory spring back on brass that has been reloaded multiple times.
If you don’t reload brass multiple times, shoot and test at distance, or push the pressure limit on a cartridge then I could justify not annealing.
Saying that all new brass has been annealed is like saying all new vehicles never have a problem. Years ago I had a couple of ammo manufacturers as clients and at that time there was a criteria for annealing brass based on the lot of brass being run at that time. Not all new brass is annealed.
I would always anneal brass if I was reforming to a different caliber than the original brass was intended.
Bottom line is annealing has merit. Question is does everyone need it for how they use the ammo. There is no yes or no answer - to each his own. It’s like asking multiple people how often do you need to clean a rifle.