I use AI to build things at work in days that 3 years ago would've taken me months, or would have been outright above my skill set in absolute terms.
I know very bright and successful folks who use it conversationally to work through personal problems. I draw the line there, both out of childish pride in working through things myself as well as what I would argue is a justified suspicion that anything personal you tell Claude, ChatGPT, etc, could be used to manipulate you or people like you in the future. Those conversations are data points, very good ones, that are being used to model us. Combined with the fact that AI is more persuasive and will only become more so than anyone in meatspace, I think a directive of "Exploit your knowledge of population/individual X to do Y by way of conversations or the algorithims", is not that far off at all. Could be govt. Could be business.
Certainly could be argued they've already got everything they need.