The problem with (some) psychological questionnaires

 

Many psychological measures and questionnaires are well designed and thought out. But some are not. Their questions can be interpreted in different ways which leaves the answers open for misinterpretation. Or they don’t measure everything needed to give a good picture of the situation.

For instance, here are my answers to two different measures of depression. A short one (MASDR) where the questions are reasonably well phrased. And another (CES-D) where they ask about the frequency of specific types of experience, but not the intensity. This provides insufficient data for any useful interpretation, and – again – leaves the answers open to misinterpretation.

MASDR

  1. Apparent sadness?
    1 – no sadness / looks dispirited but brightens up
  2. Reported sadness?
    1 – occasional sadness in keeping w circumstances
  3. Inner tension?
    1-2 – I do experience low level psychological tension. (Which is useful since it points to what’s left.)
  4. Sleep?
    0-1 – good sleep although when I go to bed early because of physical tiredness it may take a few hours to sleep solidly (because I go to bed earlier than normal)
  5. Appetite?
    0 – very good
  6. Concentration?
    4-5 – not so good because of the CFS brain fog
  7. Difficulty getting started?
    4 – also not so good because of the CFS fatigue and brain fog (there is no lack of willingness or motivation but the CFS sometimes stops me)
  8. Inability to feel?
    1-2 – reduced ability to enjoy usual interests because of CFS and brain fog (I still enjoy it as much or more than before but cannot engage physically as much as before due to the CFS)
  9. Pessimistic thoughts?
    2 – also because of CFS, realistic concern about my health and future
  10. suicidal thoughts?
    Close to 0 – as a very brief flicker, a question with no energy behind it

Summary of MASDR

The items on the MASDR scale are answered on a scale from 0 to 6, with 0 as not impacted at all and 6 strongly impacted.

If I include answers directly influenced by the CFS symptoms, then my sum is 14-18 which is interpreted as a mild depression.

If I exclude answers directly influenced by the CFS symptoms (#6 & #7), the sum is 6-10 which is interpreted as no depression to low mild depression. I would say that’s accurate. I don’t think I experience actual depression but sometimes experience some frustration and sadness in response to specific situations created from the health problems.

CES-D

Note: Since this scale leaves out intensity, I’ll add an intensity scale from 0 to 3 (to match their existing four-point scale). This intensity scale should really be differentiated into (a) strength of experience and (b) how serious I take it or how identified I am with it.

1. I was bothered by things that don’t usually bother me.

5-7 days because of hypersensitivity from CFS / brain fog – light sensitivity and sound sensitivity. For instance, I am more bothered by lawn movers and other loud sounds than I would without the CFS and brain fog. This is still at a low level, but enough to slightly notice most days.

Intensity: 1

2. I did not feel like eating; my appetite was poor.

<1 day – my appetite is generally very good. (Although I could say 5-7 days because I tend to intentionally do mini-fasting for a few hours and don’t feel like eating during those periods.)

Intensity: 0-1

3. I felt that I could not shake off the blues even with the help of my family or friends.

<1 day – this is phrased so my answer is more likely to be interpreted correctly.

Intensity: 0-1

4. I felt that I was just as good as other people.

5-7 days – yes, I can find that each day. (And I can, of course, find thoughts about the reverse as well.)

Intensity: 3

5. I had trouble keeping my mind on what I was doing.

5-7 days because of the CFS  and brain fog. Lack of frontal cortex function is a typical symptom, and one that I have to some extent all the time and strongly at times almost each day.

Intensity: 2-3

6. I felt depressed.

5-7 days – yes, I can find a small trace of it at any moment, a very slight sadness over having CFS / brain fog

Intensity: 0-1

7. I felt everything I did was an effort.

5-7 days bc of CFS / brain fog, always takes a little more effort than without it.

Intensity: 2-3

8. I felt hopeful about the future.

5-7 days bc can always find that, hopeful, excitement

Intensity: 2-3

9. I thought my life had been a failure.

5-7 days because I can find that too, a tiny thought or question (although don’t consciously see it that way at all, feel I have had a very full and rich life)

Intensity: 0-1

10. I felt fearful.

5-7 days. Yes, can find a tiny amount of it each day. (Almost imperceptible, no practical impact.)

Intensity: close to 0

11. My sleep was restless.

<1 day. I had one morning with slightly unsettling and restless dreams. (Most likely connected to what I ate the night before.)

Intensity: close to 0

12. I was happy.

5-7 days. Yes, each day and every moment. Even when other emotions or states are more prominent, there is still an undercurrent of joy and excitement.

Intensity: 2

13. I talked less than usual.

5-7- days. Yes, in some situations I talk less than usual because of CFS and brain fog. Especially if I am extra exhausted and I am in a group so I can leave it to others to keep a conversation going.

Intensity: 0-1

14. I felt lonely.

5-7 days. I can find that too. (Although consciously and overall I very rarely feel that way.)

Intensity: 0-1

15. People were unfriendly.

<1 day. I assume the question is asking if I experience people as unfriendly to me. I don’t notice it in my life but I see it in the world now and then.

Intensity: close to 0

16. I enjoyed life.

5-7 days. Yes, I overall enjoy life very much and I can find it in each moment.

Intensity: 3

17. I had crying spells.

0 – this is easier to answer as I can’t even remember last time I cried. (I do sometimes get moist eyes when I watch a movie, but that’s about it. I have probably internalized the “no crying” norm for boys, although I haven’t felt any need for crying for a long time.)

Intensity: 0

18. I felt sad.

5-7 days – yes, as with most other questions here I can easily find it in each moment. It’s there as a tiny trace. (But not as an overall or very prominent feeling.)

Intensity: 0-1

19. I felt that people disliked me.

<1 day. Not in my everyday life. Although if I want I can imagine some disliking me. (For instance my ex-wife who was upset when we got divorced.)

Intensity: close to 0

20. I could not get “going”.

5-7 days. This is a typical symptom of CFS and brain fog, and something I experience daily. The motivation, wish, and plans are there, but fatigue and brain fog very much limits what I can do, and what I can do without having to pay for it later.

Intensity: 2-3

Summary of CES-D

Answering the CES-D this way, which is the most – and only – honest way I can answer, would give a very high score. And if taken at face value, it would be very inaccurate and misleading.

Since my teens, I have been interested in and acutely aware of minute fluctations of states and experiences. Typically, all are there in trace amounts. And occasionally, one is stronger and the others are still here in trace amounts.

In addition, I would think that just about everyone experience a mix of joy, sadness, excitement, fear, anger etc. through any one day, even if many of these are minute. Not everyone notices. But some do, and then questionnaires that leave out intensity do not work very well. Accurate answers give a very misleading result. And to get a more accurate result, we would have to dumb down quite a lot when answering the questions.

I should add that even an intensity scale can be misleading, depending on how aware we are. As mentioned earlier, it should really be differentiated into (a) strenght of the experience, and (b) how identified we are with it. Sometimes, I have strong experiences I am not identified with. And other times, I can be identified with very subtle experiences. And those two are very different.

Also, as with the MASDR questionnaire, some questions inadvertingly measures symptoms of CFS instead of depression.

Personal experience. Poorly designed psychological measures and questionnaires have been a pet peeve of mine since my teens. But it’s only recently it has directly impacted me. I have gone through examinations at the CFS-Center in Oslo in order to receive an official diagnosis (I already have a CFS diagnosis from clinics abroad and my regular doctor.)

As part of the examination, I was given the CES-D (?) questionnaire and had a conversation with a psychologist. I answered it much as I did here fully knowing it would probably be misinterpreted. After looking at it, and without going through the questionnaire with me, she was convinced I had severe depression and anxiety. I thought it was surprising that she didn’t go through it with me. If she had, it would have been very clear that although I gave a high score on the frequency, the (missing) intensity score would be very low for most questions.

It seems that based on her interpretation of the questionnaire, she decided I was severely depressed and have strong anxiety, and she filtered everything from then on through that assumption. She then passed that view on to the main doctor at the center, and I had the same experience there.

Fortunately, they referred me to a specialist psychologist for treatment. Within minutes he said “Why are you here? Why did they refer you? You are clearly not very depressed and clearly don’t have much anxiety.” So hopefully, that will clear it up. (Although based on my experience with the CFS center, I am not sure.)

In hindsight, it would have been better for me to play dumb and answer the way most would have. And then told the psychologist I thought the questionnaire was not well designed since it left out questions about intensity. Alternately, I could have answered accurately and insisted we went through the answers together. The reason I didn’t do either is that it didn’t seem important at the time. I have received CFS diagnosis from specialist clinics abroad as well as my main doctor in Norway, so I somehow thought that would be sufficient.

If I am honest, I am amazed that this is even an issue. The CES-D questionnaire is clearly badly designed since it leaves out the question of intensity. I am surprised it’s used, and even more surprised that when the CFS-Center still chose to use it, they don’t ask for the intensity levels afterwards to build up a better picture.

Note: An argument for CES-D and similar tests may be that they are shown to have validity etc. They may well work for most people since they will answer yes if something was strongly present a certain day. But it does not work for people who are more aware of their experience, and – like me – will answer yes if it’s there even as a trace. (Mainly because anything else would mean using an arbitrary cut-off point.)

Related posts

Leave a Reply

Your email address will not be published. Required fields are marked *