Abstract

Background: Interest in the self-perception of patients with mental illness is increasing because of the gap between the physician and patient perceptions of symptoms and drug side effects. Schizophrenia is one of the mental illnesses that cause the greatest difficulty in understanding the various physical and psychological impacts related to both symptoms and drug treatment. Thus, several patient-reported outcomes (PRO) have been developed to get a clearer understanding of patients’ experience with their own illness and the treatment that they receive. Objective: The aim of this study was to identify all PRO questionnaires used in the evaluation of patients with schizophrenia, and to assess the quality of these questionnaires based on psychometric evidence. Methods: All PRO questionnaires used in the evaluation of patients with schizophrenia were identified using a search strategy in Pubmed, Medline, and the ISPOR website. PRO psychometric properties were extracted, and their level of validation was assessed. Results: The literature search resulted in the identification of 70 generic, mental health-specific or schizophrenia-specific instruments. These questionnaires were categorized according to measured domains. Six major domains were identified: health related quality of life (HRQoL), insight, depression/ feelings, treatment related, illness symptoms, and caregiver/family. Questionnaires measuring other dimensions were classified as other (related to personality measurement, communication between patients and clinicians and services satisfaction). The review shows that the HRQoL questionnaires demonstrate the best psychometric properties. Conclusion: The assessments of these questionnaires, based on their psychometric evidence level, will allow researchers to choose the most appropriate PRO instruments, based on the instruments’ ability to respond to the objectives of the study and on the rigor of their psychometric qualities. As schizophrenia is a multidimensional mental illness, we argue that a single PRO questionnaire is insufficient to obtain a clear understanding of the condition and treatment effects in patients with schizophrenia. Thus, we think that new PRO instruments in schizophrenia should be developed.

Keywords

Schizophrenia; PRO; Validation; Psychometric Properties

Share and Cite:

1. Introduction

Schizophrenia is a chronic, severe, and disabling brain disorder [1]. Worldwide, approximately seven per thousand adults (15 - 35 years old) develop schizophrenia during their lifetime [2]. The severity of the symptoms and long-lasting, chronic pattern of schizophrenia often cause a high degree of disability.

Three large concepts define the schizophrenia’s symptomatology: positive symptoms, negative symptoms, and cognitive symptoms [3]. Positive symptoms include hallucinations, delusions and thought disorders. Negative symptoms are symptoms associated with disruptions to normal emotions and behaviors, such as lack of pleasure in everyday life or lack of ability to begin and sustain planned activities. Cognitive symptoms include poor executive functioning, trouble focusing or paying attention, and problems with working memory, and can cause great emotional distress.

Many hypotheses were generated in order to understand the causes of schizophrenia, but no single approach was sufficient. Thus, the development of treatment primarily aimed at elimination of symptoms [3]. Treatments include antipsychotic medications and various psychosocial therapies. Compliance with therapy allows the reduction and control of symptoms. However, it doesn’t totally suppress them, and some residual symptoms have to be endured by patients with schizophrenia for a lifetime [3].

Multiple questionnaires exist to assess treatment benefit for patients with schizophrenia [4]. In a context where physicians and payers consider not only evidence-based information, but also patient-oriented criteria, it is necessary to distinguish between questionnaires answered by physicians also called clinician-reported outcomes (ClinROs), patient reported outcomes (PROs) and observer-reported/caregiver-reported outcomes (ObsROs), to capture a complete understanding of this patient population. According to the US Food and Drug Administration (FDA) definition (2010) [5], PROs are what patients report directly about their health condition, without any clinical interpretation by physicians or researchers. PROs present several advantages. First, PRO data are gathered from the information given directly by patients, without any clinical interpretation (improvement in clinical measures may not correspond to improvement from a patient’s point of view) [4]. Second, some treatment effects are only perceived by patients and hence cannot be measured by physicians (e.g. pain) and are not accompanied by physical signs. Third, patient-rated outcomes engage the consumers/patients in their mental health care [4]. However, PROs in schizophrenia face a major limitation: psychiatric symptoms, such as hallucinations and delusions, may impact the patient’s judgment and make it unreliable. The issue of insight may also be a hurdle. In spite of the potential difficulties in obtaining outcomes information from patients with certain types of mental illness, evidence suggests that even among those with chronic and severe mental illness, patients are able to evaluate their condition, and the information they provide is unique and invaluable [6]. A recent study has demonstrated that it is feasible to routinely collect mental health outcomes data from patients with schizophrenia in maintenance phase [7]. Although there is important ongoing clinical debate regarding whether or not PROs should be used in clinical trials to assess treatment efficacy, PROs in schizophrenia are widely used. Another generic limitation of PROs is that they are considered time-consuming, and that patients regard data collection as a burden [8]. They may suffer from what is called “survey fatigue” and become reluctant to fill in questionnaires.

There exist over 3000 generic and disease-specific PRO instruments [9], and some are now widely used in research contexts. With the rising use of PROs in clinical settings or to support decision-making, researchers are now facing the issue of which instruments to choose. As such a review of the evidence concerning PROs in the complex field of schizophrenia would be beneficial. The aim of the present review is to conduct a systematic literature review to identify all PRO questionnaires used in the evaluation of patients with schizophrenia, and to evaluate the quality of these questionnaires based on their psychometric evidence directly related to samples of patients with schizophrenia.

2. Methods

2.1. Search Strategy

A computerized literature search was performed using Pubmed, Medline, and the ISPOR website (Table 1). The search was conducted by two reviewers. Additional references were selected through searching the references cited by the identified studies.

There were no language restrictions, as translations were done when required.

The two reviewers independently assessed the titles and abstracts of all collected publications for possible inclusion in the study; disagreements between reviewers were resolved by consensus.

2.2. Selection Criteria

Specific criteria guided the selection of articles and abstracts discussing the different PRO instruments. Based on the FDA definition of PRO, we selected only the patient-rated questionnaires. We extracted the target popu-

lation (generic or specific), the dimensions, languages in which they are developed or translated, and the number of items. Also, we specified whether the article retrieved was dedicated to the PRO validation (described in the abstract) or only mentioned the PRO level of validity (not described in the abstract). Finally, we extracted psychometric properties such as reliability (internal consistency and reproducibility), validity (content validity and construct validity) and sensitivity to change. Table 2 defines these parameters.

Reliability, validity and sensitivity to change were assessed by two trained analysts having at least 3 years’ experience in the field. Analysts independently rated the level of reliability, validity and sensitivity to change as robust (++), moderate (+) or poor (−). Basically, analyses were considered robust when all evidence was provided in the publication, and suggested a high quality, as moderate when only part of the evidence was provided, and poor when not all analyses were performed, or when the level of evidence suggested a low quality.

The rating was based on the number and the kind of the analyses run, and obviously on their results. For example, if only the internal consistency is assessed but no test-retest analysis is performed, the reliability could not be rated as robust, as only one aspect of reliability was tested. When no quantitative results were reported in the publication to support authors’ conclusions, analysts stated “not detailed”. In other cases, analysts stated “not assessed”.

3. Results

3.1. Overview of Studies

A large number of PRO questionnaires have been developed for patients with schizophrenia. A total of 177 abstracts were selected (114 from Embase/Medline, 51 from ISPOR and 12 from other sources). Excluded were 107 references that did not present any PRO for patients with schizophrenia, and finally 70 references were selected in the present review. As outcomes in schizophrenia may be divided into several categories that are largely independent of each other, 10 domains were defined from the selected questionnaires, namely quality of life, depression/feelings, insight, treatment-related, illness symptoms, personality measure, social functioning, communication patient/clinician, services satisfaction and PRO related to caregiver and family burden. Below, we regroup PRO questionnaires according to their domains.

Although all psychometric characteristics were extracted from the 70 references, they could not be fully reported in this article. They are available on request.

3.2. Health Related Quality of Life Questionnaire (HRQoL)

Quality of life it is a complex multidimensional concept. It provides information about the patient’s evaluation of life in its both positive and negative aspects. Several domains are included in the quality of life concept, such as job, housing, school, aspects of culture, values, spirituality, etc. Health is one of the most important dimensions of the QoL concept [10,11]. HRQoL measurement has become an important outcome both in health assessment and in judging clinical improvement [12,13]. Health care community argues that individual’s health is a multidimensional concept including physical, mental and social domains [14]. It can also include emotional and behavioral components of well being and functioning. HRQoL represents, then, the patient’s perception of his physical and mental health condition and of the functional impact of illness on them. In recent years, instruments to measure HRQoL in mental health have been progressively incorporated as a means of measuring the effects of interventions on symptoms and functioning.

Table 3 presents the 18 PRO instruments retrieved from our literature review that were used to evaluate HRQoL in schizophrenia.

The TOOL [15-18], a specific measure to assess

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

HRQoL impairment related to adverse events of antipsychotic drugs using a 8-item specific scale, demonstrated the strongest psychometric properties. The SWN questionnaire [19-21], like the TOOL, shows a high validity. However, reliability and sensitivity to change are less validated than those of the TOOL. The Schizophrenia Quality of Life Scale Revision 4 (SQLS-4) is the result of the several revisions made to the SQLS to improve its psychometric properties. The SQLS-R4 contains 33 items in two domains: psychosocial (20 items) and vitality (13 items). The Q-LES-Q-18 is a core subset of the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q) items that maintains the validity and psychometric properties of the basic version. It has the advantage of a 10-12 min administration time, compared to the basic Q-LES-Q, which took 40 - 45 min to complete. The EQ-5D and the SF-36 are widely used, and allow calculation of patient utility.

3.3. Insight PRO Instruments

Definition of insight has much evolved since recent decades: from a “patient’s awareness about a mental disorder that is either present or absent” [22-24] to“patient’s awareness that he or she is suffering from a mental illness, experiencing symptoms of that illness, in need of treatment, and that the cause of symptoms is the illness” [25,26].

Despite “insight” being primarily a clinician’s concept, there exist several self-rated instruments.

Four Insight PRO instruments were selected in this present study (Table 4). With no assessed sensitivity to change, the ABPS [27] shows high validity and reliability. The BCIS [28] has a high validity but reliability and sensitivity to change were not assessed.

3.4. Emotional Well Being

Learning how depression, stress and anxiety may affect patients with schizophrenia is one of the major concerns of clinicians nowadays. Depression is a frequently occurring symptom in schizophrenia. It is often under-recognised and under-treated. Few PRO instruments aim to measure symptoms of depression, or other feelings like anger.

Table 5 presents five depression/feeling PRO instruments, but no one has the psychometric properties required. Indeed, in the majority of these PRO instruments, neither the validity nor the reliability was assessed in the population of patients with schizophrenia.

3.5. Treatment-Related PRO Instruments

Neuroleptic medication is the most widely used treatment for schizophrenia. It is effective in reducing psychotic symptoms, but compliance is often poor because neuroleptic drugs are associated with a wide range of side-effects. Medication-related PRO instruments can measure different variables related to treatment: the response to the medication, the adherence or compliance to the treatment, or the side-effects.

Eleven treatment-related PRO instruments were selected (Table 6). In all of these PRO instruments, the sensitivity to change was not assessed. The LUNSERS [29], a 51-item specific scale, shows high validity and reliability.

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

3.6. Illness Symptom PRO Instruments

Fifteen illness symptom PRO instruments were selected.

Table 7 shows that sensitivity to change was not assessed for all these PRO instruments. Only the WHODAS-II [30] 32-item scale has high validity and reliability. PNS-Q [31], TEPS [32,33], RAS [34] and ReSUS [35] have high validity and average reliability. Only SHAPS [36] presents average validity and high reliability. The PPS scales [37-40] present average reliability and validity.

3.7. Caregiver or Family PRO Instruments

Successful treatment of schizophrenia often involves the patients being supported in their day-to-day functions by a family member or caregiver. This role carries a heavy responsibility but has received a disproportionately low level of attention. The level of support required varies widely from patient to patient, depending upon the severity of disease symptoms; for example, those with minimal levels of cognitive functioning often require fulltime supervision.

Families and caregivers need access to a support network, as their role in the patient’s recovery is often long term and can be quite demanding physically, emotionally, financially, and in terms of time.

In this review, authors considered that instruments assessing caregiver’s burden were PROs and not ObsROs, as they describe the respondent’s burden, not the patient’s burden.

Eight caregiver or family PRO instruments were selected. Table 8 shows that sensitivity to change was not assessed for any of these questionnaires. Both the ECI [20,41], a 66-item mental illness-specific scale, and the IEQ [42], a 31-item schizo-specific scale, have high validity and reliability. Three other questionnaires present high validity and average reliability: the S-CGQoL [43], the PFBS [44] and the FAS [45,46].

3.8. Other PROs for Patients

Other questionnaires may also be used in the population of patients with schizophrenia. Table 9 includes twelve PRO instruments that are classified according to measured dimensions. The dimensions measured in these twelve questionnaires are: personality, social functioning, communication (patient/client), service satisfaction, and overall knowledge and understanding of schizophrenia. Both validity and reliability are high for the CSQ [47], an 8-item generic scale. The sensitivity to change was, however, not assessed. Both the SFS [48-50] and the WRSE-38 [51], a schizo-specific scale, show high reliability and average validity and sensitivity to change. The PAI [52] has high validity and average reliability. Sensitivity to change was not assessed for this questionnaire. The KASQ [53], a 25-items specific scale, shows acceptable psychometric properties with a high reliability and average validity and sensitivity to change.

4. Discussion

Health Technology Assessment (HTA) is intended to provide a bridge between the world of research and the world of clinical decision-making [54]. Today, both worlds are willing to better involve the patient, to account for patient’s perspective on the same level as the clinician’s perspective. Willke et al. report that PROs have a significant role in the development and evaluation of new medicines [55]. Hence, PRO instruments were developed to bring a clearer comprehension of the patient’s self-described status. Indeed, “many aspects of

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

patients’ experiences with illness, medication, and health care are best captured from PROs” [56].

The present study identified 70 PRO instruments that are used in patients with schizophrenia. Their quality assessment was based on a number of essential properties that define a high-quality instrument.

Moreover, a correctly developed PRO should satisfy psychometric properties: it has to be reliable and valid (including responsive to underlying change). Content validity is a critical aspect to be considered in that context [5]. Also, construct validity (whether the structure of the scale includes a single or multiple domains) should be thoroughly tested using appropriate methodology in order to justify the use of scale or summary scores. These properties were the basic criteria in our assessment of PRO instruments presented here.

In addition, these standards must be maintained throughout every target language population. In order to ensure that developmental standards are consistent in translated versions of a PRO instrument, the translated instrument undergoes a process known as linguistic validation in which the preliminary translation is adapted to

++: Robust; +: Moderate; - Poor; NA: Not Assessed; ND: Not Detailed.

reflect cultural and linguistic differences between diverse target populations.

We found that the TOOL is the most validated HRQoL PRO and appears to be the most appropriate scale for patients with schizophrenia, even if it is specific to a wider population (severe mental illness population). In addition, the SQLS-R4 has been translated into 52 languages through standardized procedures (i.e., forward translation, reconciliation, and back translation), which makes it a very interesting tool for large international comparison.

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

The BCIS seems to be the most widely used scale to assess patient insight, even though it was not evaluated for reliability and sensitivity to change. The BDI and BDI-II are used to assess depression for patients with schizophrenia, but the validity of the scales was not assessed for this population in particular. The LUNSERS questionnaire is the most widely used and validated questionnaire to evaluate side effects. The PPS is a complete set of scales to measures symptoms of schizophrenia, but it is very large; it is composed of 5 scales with several dimensions themselves. In practice, only a part of this battery of scales is used, depending on the objective of the study.

It is noteworthy that no instrument has been developed in line with FDA requirements: when developing a PRO instrument, sponsors should explain the development process of several concepts: items generation, choice of the data collection method, choice of response options, evaluation of patient understanding, development of format, instructions and training, identification of preliminary scoring of items and domains, assessment of respondent and administrator burden, confirmation of the conceptual framework and finalization of the instrument [5]. Despite the high number of PROs identified in this study, qualitative and quantitative analyzes available for these scales tended to significantly reduce the number of well-validated questionnaires. Nevertheless we recommend that future researchers pay attention not only to the basic critera mentioned above, especially high psychometric properties, but also to the ability of PRO instruments to provide the desired information, and to the application for which the instrument is intended (research, evaluation, individual patient care or population assessment).

A limitation of this study was the lack of exhaustivity of the publications: most articles selected via the search strategy were retrieved and extracted, but some of them were not available at the time of the analysis, and only abstract information was included.

In addition to the particular attention needed around psychometric proprieties, our recommendation for new PROs to be developed includes the necessity to demonstrate a relative independence from existing instruments. It is of importance to identify broader PRO outcomes that are relevant to clinicians and key stakeholders, identifying the best possible measures to assess these, capturing treatment benefit from the patient perspective so that patient, clinicans and key stakeholders gain a better insight to the risks and benefits of new treatments.

5. Conclusions

The aim of this review was to identify all PRO questionnaires used in the evaluation of patients with schizophrenia, and to evaluate the quality of these questionnaires based on their psychometric evidence.

The results show that the HRQoL PROs are the most

++: Robust; +: Moderate; −: Poor; NA: Not Assessed; ND: Not Detailed.

numerous and the best validated in schizophrenia. Although selection of PROs should be based on the rigor of their psychometric properties, researchers should also focus on their ability to respond to objectives of their study.

As schizophrenia is a multi-dimensional mental illness, we argue that today, the validity of PRO questionnaires is insufficient to obtain a clear comprehension of the condition and treatment effects in patients with schizophrenia. Thus, we think that new PRO instruments in schizophrenia should be developed in order to reach the full spectrum of possible outcomes related to the disease and its treatment.

NOTES

Conflicts of Interest

The authors declare no conflicts of interest.

References