I think you've nailed it. Social smoking is ok because I've got lots to distract me from my thoughts. The last few times I smoked here in Spain (more than a year ago mind you) I did with friends at a BBQ or whatever and it was fine.
Smoking by myself, left alone with my thoughts, especially sat in front of a computer, is bad news. Have to get up, walk away, and talk myself through it staring at the sea and relaxing.
It used to be great to smoke a joint and sit down for 5 hours of grinding away on a project. I wish I knew what the trigger was for what happened to me, because I used to really enjoy it. Where did I lose control?
This happened to my friend also. He smoked for like 13 years and then developed anxiety (not from smoking weed). Once he got anxiety, everytime he smoked, it made the anxiety worsen. He would get all tense instead of relaxed and stoned. I think if you can get yourself back to the state of mind you were in before the anxiety by correcting whatever caused it in the first place....you can get ripped and enjoy the relaxation again....but who knows. I have heard of a lot of long time smokers stopping for this reason as they get older.