Using Quality of Life Measures for Program Evaluation: A Review of the Literature

by Caroline Claussen

Trying to articulate "the good life" has preoccupied many great philosophers throughout the centuries. Defining and measuring quality of life is currently at the forefront of the rehabilitation field. However, with all the limitations on the concept, many people ask "Why study quality of life at all?" Some question the use of this concept in evaluation of programs and services.

The concept of quality of life [QoL] is relatively new, having only been used to describe population conditions since the 1960s (Wolfensberger, 1994). It is a concept that is often misunderstood and misapplied. Not only do different disciplines define it different ly, but the issue of cross-cultural measurement also affects a stable and lasting definition. Despite the difficulty of defining and measuring QoL, many feel that the importance of the effort is obvious given that the findings from QoL research are directly relevant to fundamental concerns of societies and individuals.

Through the 1990s and into the 21st century, quality of life has become the buzz word, with evaluation of service delivery becoming a central focus for policy makers and funders. Many studies have used QoL measures to guide program and rehabilitation efforts. This review will discuss the concept of measuring 'quality of life', and address the use of QoL indicators for the evaluation of programs and services.

Quality of life is a vague term; it means different things for different people. With all the limitations on the term (e.g., conflicting definitions, and cultural limitations), researchers are still attempting to define and measure it. During the last two decades, two main scientific approaches have been initiated: the use of "objective," or social indicators, and the measurement of subjective well-being (Diener & Suh, 1997).

Objective measurement is achieved through the use of social indicators. These are societal measures that reflect people's objective circumstances in a given cultural or geographical unit (Diener & Suh, 1997). The use of objective, quantitative statistics is the hallmark of objective assessment. General indicators of QoL are established by an assessment of a variety of life conditions across the population, such as social welfare, education, infant mortality, standard of living, crime rates etc. The measurement focuses on key aspects of life that can be improved, such as the degree to which basic needs are met, and the degree of material and social attainment (Schalock, Brown, Cummins, Felce, Matikka, Keith & Parmeter, 2002).

Objectivity is one of the major strengths in using social indicators to assess QoL. The indicators can be relatively easy to define and quantify without relying on individual perceptions (Diener & Suh, 1997). However, many argue that QoL cannot be measured from external factors because it is all about individual experiences (Wilk, 1999), i.e., that objective indicators may not be reflective of people's experiences of well-being (Diener & Suh, 1997). There are those who argue that there is no such thing as objective indicators, since they are based on the subjective opinion of those experts who deem them worth measuring.

More and more, researchers are recognizing the need to "hear" the voices of people with developmental disabilities. Subjective measurement attempts to do just this, concerning itself with respondents' own internal judgment of well-being, rather than what policy makers, academics or others consider important (Diener & Suh, 1997).

The major advantage of subjective QoL measures is that they capture experiences that are important to the individual (Diener & Suh, 1997). Most social indicators are indirect measures of how people feel about their life conditions, whereas subjective measures provide important additional information that can enhance and validate the data provided by objective indicators. Also, subjective measures tend to correspond more closely to people's value systems than objective measures do.

However, the most important weakness of subjective measurement is that they may not fully reflect the objective quality of community life in a location or population (Diener & Suh, 1997). For example, people may report having a high life quality even if they are in poor health or live in absolute poverty. This result may be due in large part to individual temperament and personal relationships.

Most researchers agree that the use of both objective and subjective measures provides the best overall picture. There are a number of tools that incorporate both aspects, e.g., the Quality of Life Questionnaire, by Schalock & Keith (1993), and the Comprehensive Quality of Life Scale by Cummins (1997). Both tools use objective and subjective measures across a range of domains, provide a definition of life quality, have clear administration and scoring procedures, and yield some psychometric data. The Schalock and Keith questionnaire has been used extensively by researchers and service providers alike, while the Comprehensive Quality of Life Scale has been undergoing revisions to make it more widely available (Cummins, 1997).

There is a popular trend in the rehabilitation field of using subjective QoL measures to evaluate services and programs. However, there are those who contend that subjective QoL measurement is not the best way to determine service quality (Hatton & Ager, 2002; Wolfensberger, 1994). They advance several arguments to make their case.

The first argument is that consumer perceptions of quality are only partially the result of service program structure (Wolfensberger, 1994). Every person receiving service is a different individual. It is very possible that two people under identical conditions will self-report differently. As Wolfensberger points out, some individuals will be unhappy no matter what the circumstances, while others will report being happy even under very adverse conditions.
Secondly, individuals may rate themselves as having a very high QoL, but this may not be at all the doing of the service provider (Wolfensberger, 1994). The person him/herself or other participating individuals (e.g., relatives) may be contributing to the high life quality. A very positive self-report cannot be assumed to be solely the result of a particular program or of good service (Hatton & Ager, 2002). One of the ways to attribute high QoL to service is when an individual is in a total environment (i.e., an institution). In community-based programs, it is very difficult to attribute high life quality to excellent service when there are so many other possible influences on the individual (e.g., family, friends, individual temperament, activities outside the program etc.). In a, institutional environment, the same service model surrounds the individual 24 hours a day at a single location. Although the potential for outside influence exists where an individual receives regular visits, it is less than for those who live in community settings without a 24-hour service model.

Finally, Wolfensberger (1994) points out that individuals may report a high QoL because they do not know any different. For instance, people who have never known life outside of a poorly run institution may report having a high life quality. Objectively poor life circumstances can be justified by apparently high service user satisfaction (Hatton & Ager, 2002).

There is no doubt that quality of life for people with developmental disabilities should be of the utmost concern for service providers and staff. But quality is not always so easy. Using both subjective and objective measures will allow us to provide a complete picture of life quality, as opposed to preferring one measure over another. There are convincing arguments against using QoL outcomes to determine service quality. Careful consideration needs to be given to the assumption that if QoL standards are met, then quality of service is present.

• The concept of quality of life is relatively new, dating only from the 1960s.
• Amid conflicting definitions, and cultural limitations, two main scientific measures have emerged: “objective,” or social indicators; and subjective perceptions of well-being.
• “Objective” approaches may be nothing more than the subjectivity of the researchers in deciding what to measure.
• Conversely, people may subjectively report high life quality even if they are in poor health or live in absolute poverty.

• The assumption that if quality of life standards are met, then quality of service is present, merits careful consideration.

Cummins, R.A. (1997). Self-rated quality of life scales for people with an intellectual disability: A review. Journal of Applied Research in Intellectual Disabilities, 10 (3), 199-216.
Diener, E., & Suh, E. (1997). Measuring quality of life: economic, social, and subjective indicators. Social Indicators Research, 40, 189-216.
Hatton, C., & Ager, A. (2002). Quality of life measurement and people with intellectual disabilities: A reply to Cummins. Journal of Applied Research in Intellectual Disabilities, 15, 254-260.
Schalock, R.L., Brown, I., Brown, R., Cummins, R.A., Felce, D., Matikka, L., Keith, K.D., & Parmenter, T. (2002). Conceptualization, measurement, and application of quality of life for persons with intellectual disabilities: Report of an international panel of experts. Mental Retardation, 40 (6), 457-470.
Wilk, R. (1999). Quality of life and the anthropological perspective. Feminist Economics, 5 (2), 91-93.
Wolfensberger, W. (1994). Let’s hang up ‘quality of life’ as a hopeless term. In D. Goode (Ed.), Quality of Life for Persons with Developmental Disabilities: International Perspectives and Issues (pp. 285-321). Cambridge, MA Brookline Books.