I believe they're vastly over stigmatised. That being said, I've been through addiction recovery and have learnt to control bad habits quite well, now I just do it for enjoyment or to prevent boredom, or genuinely to just control my emotions when I'm particularly in a bad state of mind.
I've touched on pretty much all types of drug from smoking, alcohol, to derivatives of m-amphetamines, have had addictive phases to both speed and coke, am a fan of opiates like morphine and codeine (lean in particular), and have tried LSD, 2cb and a bunch of other stuff a handful of times each. Obviously Xanax and Valium for months as well. Sometimes I wonder whether this was in attempt to self medicate partially since I wasn't diagnosed with ADHD until recently, so now I'm prescribed Ritalin for it and feel less inclined to "need" something to provide me excitement or anything like that.
It could also just be because I'm from quite a rural area, and most of my friends live a while away - so there's not really a whole lot else to do.
I won't lie though, with some of them like coke you just do it a few times and then it's like the best thing in the world, super difficult to stop the habit, so I wouldn't recommend it to anyone who's not already fallen into the trap themselves.
Your psychologist sounds like a bit of a dbag. A lot of people who even actively use don't want to be doing it, some are super difficult to escape - my biggest struggle in addiction was with alcohol, and still is. I could never justify alcohol over any of the others I've done, and I know it can make things just as bad, even if it's not from the first drink.
Obviously before ever doing any I was very against them and was brought up in the mindset that they were used by bad people who would rob or hurt you, or be the type to shout crazily in the streets. Not always true, but possible. I like to think I'm pretty good to be around, I just find myself enjoying them over other things since I can't find interest in much else. I've got a lot cleaner recently, however, I still drink 5 nights / 7 as I don't have much to do in lockdown and I find it improves me socially.
I have had no issue with the health sides of any of them since I don't really care for my health.