Apparently those with bipolar disorder (manic depression) can be hypersensitive to carbon dioxide triggering of anxiety and panic, according to
this study.
I also wonder if the lack of oxygen might cause anxiety and panic in some people, when using the inert gas method.
When I experimented with a homemade debreather device (see
this thread), I found that after about 3 to 5 minutes, as oxygen levels went down in my debreather air bag, I developed a strong urge to stop the test, take the mask off my mouth, and breathe fresh air — a panicky urge to breath fresh air similar to what you get when you hold your breathe for a minute or so.
This strong urge to stop the debreather test occurred when my blood oxygen saturation (measured by a finger oximeter) was around 80% to 70%. I believe my CO2 scrubber in the debreather was removing the CO2, so I think the panicky urge to stop the test was likely caused just by the low oxygen levels.