Identifying and Selecting Long-Term Services and Supports Outcome Measures


This paper provides resources to help state advocates identify measures that can help determine if the identified needs and goals of people with disabilities and seniors are being met. The need for such measures is increasingly important, as more and more states launch initiatives to provide Medicaid Long-Term Services and Supports (LTSS) — including both institutional and Home and Community-Based Settings (HCBS) — through managed care arrangements.

We focus primarily on identification of LTSS outcome measures that examine individual experience, whether that individual is the beneficiary receiving services, the beneficiary’s family caregiver, or a paid personal assistant, rather than measures that relate to structural elements or processes. This distinction is important for two reasons: 1) no validated national LTSS outcome measures are currently available, and 2) outcome measures will play a central role in generating valuable data that the Centers for Medicare and Medicaid Services (CMS), states, managed care organizations (MCO), and advocates require in order to monitor the effectiveness of managed LTSS, and craft and implement strategies for ongoing quality improvement. While the paper emphasizes outcomes for long-term home and community-based services, some individual outcomes also relate to the experiences of people living in nursing facilities.

The paper outlines the factors that have spurred the increase in states’ enrollment of Medicaid beneficiaries with disabilities into managed LTSS/HCBS and explains why measuring individual outcomes is so important as these changes are implemented. Recent work to identify appropriate individual outcome measures as well as gaps in needed measures are identified and discussed. Examples of measures that are in development and that are in current use are also presented.

We present core principles and criteria for selection of LTSS outcome measures. We also present possible sources of data that could be used to respond to specific questions related to quality as well as key report topics that will generate information needed for quality improvement. The paper also suggests methods for building LTSS knowledge and infrastructure capacity at federal agencies, states, MCOs and service providers. Included are key training recommendations for incorporating LTSS person-centered values at these various systems levels, and monitoring and reporting requirements that advocates should know.

The paper concludes with suggested action steps and resources for advocates who are working with states, MCOs, community-based service organizations, disability and senior groups and other stakeholders as states launch managed LTSS and HCBS initiatives.

Medicaid Long-Term Services and Supports, and Home and Community-Based Services: The Changing Landscape

Healthcare and LTSS mechanisms and financing for low-income people with disabilities of all ages have been undergoing significant changes in recent years. States are increasingly requiring that Medicaid beneficiaries with disabilities and seniors mandatorily enroll in managed care with the dual goals of cost saving and improved health outcomes. The 2010 Affordable Care Act (ACA) authorized a multistate demonstration to enroll people who are dually eligible for Medicare and full Medicaid benefits in managed care plans. Some of the demonstrations will also transition LTSS services into the managed care system. Slated to begin in 2013 in some states, the demonstrations aim to generate savings as well as address the gaps and instances of financial and service misalignment between Medicare and Medicaid. Moreover, beginning in 2014, the ACA will expand Medicaid to cover millions of low income, uninsured individuals, including many with disabilities. Some states that participate in the expansion will likely require that these new Medicaid beneficiaries also enroll in managed care.

Managed LTSS[1]

Historically, most states have provided Medicaid LTSS for people with disabilities and seniors primarily through fee-for-service models. Some of the methods for delivering LTSS have evolved based on core principles of self-direction and independent living espoused by disability rights advocates. Self-direction means that beneficiaries directly control a variety of services and supports – sometimes with the assistance of other individuals whom they choose – based on their own preferences and needs. For example, self-direction can mean that the beneficiary hires, supervises, and trains a personal assistance worker of her or his choice who is paid by Medicaid. The core intent of self-direction is to maximize an individual’s opportunities to live independently in the most integrated community-based setting of her or his choice.

As mandatory enrollment of low-income people with disabilities and seniors into Medicaid managed health care becomes more widespread, more states are also moving to include Medicaid LTSS as part of a managed care package. The number of states with managed LTSS programs increased from 8 in 2004 to 16 in 2012, and the number is expected to reach 26 by 2014.[2][3] Integration of LTSS with acute care in such plans typically involves a shift of responsibility for providing LTSS to MCOs that only have experience providing acute care. This model of care integration often involves including LTSS as part of a capitated payment to a traditional risk-based MCO. The term capitated payment indicates a payment method in which the managed care organization is paid a contracted rate for each member assigned, referred to as a “per-member-per-month” rate, regardless of the number or nature of services that are actually provided. The contractual rates are usually adjusted for age, gender, illness, physical and mental impairment, regional differences, and other relevant factors.

Why Are Quality Measures Needed?

As discussed, as many as 26 states may initiate managed Medicaid LTSS/HCBS programs by 2014. Many of these states will rely on MCOs that lack experience providing LTSS/HCBS and that do not have the capacity to collect timely, reliable and valid data about the care being provided, those who provide care, or consumer experience with care. This information is fundamental to all strategies for monitoring outcomes and identifying methods for improvement.[4] CMS, states, MCOs, healthcare and LTSS providers, advocates and beneficiaries require information derived from appropriate, uniform, and preferably validated measures that assess factors such as adequacy and impact of services, quality-of-life, extent of self-direction and self-determination, community integration and participation, health, functional and safety outcomes, and access to consumer rights and protections. Measuring outcomes in managed LTSS serves a variety of purposes:

  • Data enables state and federal oversight of the extent to which managed care plans’ commitments to consumer-focused and directed, quality services are being honored.
  • The presence of outcome data can encourage and enable MCOs to focus on more effectively meeting beneficiaries’ expressed needs.
  • The ability to track outcomes over time, including during the transition to integrated, managed LTSS, as well as to compare outcomes across managed care plans, gives advocates a tool to make both plans and the state accountable for appropriate service provision.
  • Data assists states, MCOs and providers to evaluate the effectiveness of integrated care coordination across both clinical and LTSS domains.
  • When consumers have a choice among plans, or whether to receive LTSS through a managed care plan, data on outcomes can help them make such choices.
  • When similar outcome measures are used across programs and service settings (e.g., community versus institutions), data can be used by consumers to make choices, and by advocates and policy makers to identify programs with the best outcomes.[5]

How Is Quality Measured?

Quality measures typically fall into three categories: structural, process, or outcome measures. Structural measures generally refer to elements of service or care, such as physical plant operations and facilities, equipment, and staff capacity. Structural elements can also include management and management structure, administration, staff qualifications and balance of professional and nonprofessional staff, data and record keeping mechanisms, and other internal quality review activities that an organization might undertake.

Process measures typically refer to the interaction between the individual who uses care and systems that provide it, and include both a technical element and an interpersonal element. The technical element measures the appropriateness of care and the capability of the provider. It also includes elements of timeliness and consistency for an intervention and the skill with which it was provided, including assessment, service planning, and provision of care or services.

Outcomes are the results of services or care. They emanate from efforts to assess and treat conditions or flow from support and services that people receive or should be receiving. Outcomes can be both beneficiary evaluation of care or support and the results of care.[6]

Quality measures in clinical settings are highly developed as compared with quality measures for LTSS, which are in the early stages of standardization and development.[7] Scientifically validated clinical quality measures in use nationwide typically include structural (e.g., physical plant operations, facilities, equipment, staff capacity), acute medical and clinical processes (e.g., hypertension or cancer screening, disease prevention such as promotion of smoking secession or weight loss), and health outcomes (e.g., lowered cholesterol or blood glucose levels, weight reduction). Fewer measures have been developed that apply to care transition for LTSS beneficiaries, that is, for care and outcomes for individuals who are transitioning between settings, such as acute care facilities, home care, assisted living, and skilled nursing care. Fewer still measure individual outcomes specifically for LTSS provided in a managed care context. To the extent that states and MCOs are attempting to measure managed LTSS effectiveness, they tend to use process measures such as whether or not beneficiaries of managed LTSS were given the choice between community-based services and institutionalization, or how frequently care needs are determined. They may also use targeted monitoring of specific process measures that are important to certain populations such as frequency of dental visits for people with developmental disabilities.[8]

While few uniform quality indicators have been tested and validated that would inform and guide monitoring and quality improvement of managed LTSS, and no national standards exist, various instruments and measures that contain many of the relevant concepts have been developed by national projects to improve LTSS outcomes and quality.[9][10][11] In light of increasing pressure to measure managed LTSS outcomes from a person-centered perspective, increased public and private efforts have focused on mapping the landscape of available measures, identifying gaps, and recommending future actions.

Moreover, CMS has taken some steps to ensure that the health and long term care needs specifically of dual eligible beneficiaries are appropriately met when they are transitioned from fee-for-service to managed care. The agency is requiring states that are participating in the dual demonstration projects to report individual level quality, cost, enrollment, and utilization data. CMS is also requiring that participating health plans report encounter data and meet certain quality indicators. However, these indicators remain to be determined. Quality indicators identified by this reporting process hold the potential to equally inform the initiatives involving mandatory enrollment of Medicaid-only beneficiaries with disabilities and seniors into managed care.[12]

Although CMS is still considering how states should measure managed LTSS/HCBS outcomes for the duals demonstrations, in 2004 the agency published a Quality Framework of LTSS/HCBS domains and desired outcomes, illustrated in Table 1, that contains key indicators:[13]

Table 1: CMS Quality Framework Domains and Desired Outcomes

Focus Desired Outcome
Participant Access Individuals have access to home and community-based services and supports in their communities.
Participant-Centered Service Planning and Delivery Services and supports are planned and effectively implemented in accordance with each participant’s unique needs, expressed preferences and decisions concerning his/ her life in the community.
Provider Capacity and Capabilities There are sufficient HCBS providers and they possess and demonstrate the capability to effectively serve participants.
Participant Safeguards Participants are safe and secure in their homes and communities, taking into account their informed and expressed choices.
Participant Rights and Responsibilities Participants receive support to exercise their rights and accept personal responsibilities.
Participant Outcomes and Satisfaction Participants are satisfied with their services and achieve desired outcomes.
System Performance The system supports participants efficiently and effectively and constantly strives to improve quality.

What Managed LTSS Outcomes Should be Measured?

The following summary of efforts thus far to identify existing outcome measures in the Medicaid LTSS context reveals both potential tools for immediate use with managed LTSS as well as significant gaps that call for additional research and practical field testing.

1. Measure Application Partnership (MAP)

The Measure Application Partnership (MAP), authorized by the ACA and convened by the National Quality Forum (NQF), has developed a national measurement strategy for the dual eligible population, which was released in June 2012. MAP separately explored quality measures in Medicaid-supported HCBS, identified such measures as a major development gap area, and recommended that HHS fund an NQF effort on quality measures in LTSS.[14] In support of this recommendation, the MAP report identified 24 potential illustrations of person-centered concepts that warrant further investigation.[15] These include unmet Activity of Daily Living (ADL) needs; degree to which people express satisfaction with relationships; degree to which people with identified health problems obtain appropriate services;[16] availability of self-direction options; and self-reported preventive health care visits. The measures fall into eleven major domains including client functioning and experience; program performance; choice of setting and provider; and quality of life and quality of care.[17] (See Attachment A.)

2. Agency for Healthcare Research and Quality (AHRQ) Scan (June 2010)

The Deficit Reduction Act (DRA) of 2005 directed AHRQ to develop quality measures for the Medicaid Home and Community-Based Services program. The DRA instructed AHRQ to develop measures in the domains of client functioning, client satisfaction, and program performance in order to assess the quality of Medicaid HCBS programs nationwide. In response to this directive, AHRQ conducted an extensive environmental scan using a broad definition of HCBS services and populations, including, for example, populations such as adults with severe and persistent mental illness who are not traditional recipients of Medicaid HCBS. Reporting research outcomes in 2010, AHRQ identified more than 200 measure sources that included survey instruments designed to yield performance measures, measure sets, and measure databases. These instruments revealed some broad themes. For example, several consumer survey tools have been developed that assess client experience with HCBS, particularly for individuals with intellectual and/or developmental disabilities. Moreover, psychometric testing has been carried out on many surveys used by several state programs. While few state-specific tools have been tested for validity, many that are in use solicit consumer feedback as a means of providing federally required assurances for Medicaid 1915(c) waiver programs. However, AHRQ also reported that no single survey tool or measure set addressed all twenty-one constructs that the Agency, with stakeholder input, had identified as applicable across all HCBS populations. (See Attachment A.)[18]

AHRQ’s eight constructs of client experience, listed below are of particular interest, however, because state-specific surveys have queried many of the same dimensions of experience: (Also see Attachment A.)

  • Respectful treatment by direct service providers.
  • Opportunities to make choices about providers.
  • Opportunities to make choices about services.
  • Satisfaction with case management services.
  • Client perception of quality of care.
  • Satisfaction and choice regarding residential setting.
  • Client report of abuse and neglect.
  • Availability of support for resilience and recovery (mental health service recipients only).

Three themes underlie these eight constructs, composing the client experience domain of the measure scan:

  • Client choice, captured in three global dimensions of program supports: providers, services, and housing.
  • The cross-cutting theme of satisfaction, represented by the queries for satisfaction with residential setting and case management services. Global satisfaction is represented by the construct for perception of the quality of care.
  • Interpersonal respect and support, which can be assessed positively, as in the constructs of respectful treatment by direct service staff and the availability of staff/program support for resilience and recovery for those with serious mental illness. The converse of positive and supportive interpersonal relationships is reflected in the remaining construct, client reports of abuse and neglect.

Overall, these eight measures can be seen as representing a continuum from harmful and unacceptable experience (e.g., neglect and abuse), through respect and individual choice, culminating in individual satisfaction.

3. The Long-Term Quality Alliance (LTQA) Quality Measurement Workgroup Report (December 2011)

The Long-Term Quality Alliance (LTQA) identified measurement gaps for LTSS beneficiaries that also suggested areas for measure development and/or research. These include transitional care measures for person- and family-centeredness. Specifically, measures were identified that contribute to overall quality for individuals and their families and that are broader than clinical outcomes (i.e., quality of life, autonomy, relationships, compassion, social supports, and emotional well-being).[19]

4. Center for Personal Assistance Services, University of California San Francisco

  1. In a March 2012 policy document that summarized the results of a review of the literature, the Center for Personal Assistance Services suggested that in order to ensure managed care systems provide appropriate and effective LTSS for newly enrolled low income individuals with disabilities of all ages, a uniform set of LTSS outcome measures must be identified with input from consumers, advocates, and other stakeholders. The Center urged that stakeholders be afforded a variety of measures from which to choose that reflect their values rather than reflecting the particular services they are receiving. The Center accordingly recommended five overarching areas that include such measures, as well as measures of broader outcomes related to the beneficiary, his or her family, and the informal caregivers and paid workers who provide services.[20] (See Attachment A.)
  2. Selected Inventory of Quality-of-Life Measures for Long-Term Services and Supports Participant Experience Surveys.[21]

To begin to address the gap in methods to measure LTSS quality of life outcomes, the Center, using Wisconsin’s Personal Experience Outcomes Integrated Interview and Evaluation System (PEONIES) (See 6c below) domains as a starting point, searched among relevant, existing survey instruments to identify previously field tested questions related to these domains and to quality of life (QOL) measures. This research yielded a list of measures that might either be

used or adapted to construct concise surveys useful for monitoring particular programs serving specific populations. (See,

5. CMS HCBS Quality Measurement Projects Under Development[22]

  1. Money Follows the Person (MFP) Quality of Life Survey (QoL)

The Center for Medicaid, CHIP and Survey & Certification (CMCSC) called for development of The Money Follows the Person Quality of Life Survey (QoL), which is slated to be tested in 2012. The target population for the survey includes people with disabilities and long-term illnesses who are transitioning from institutionalized care to a care setting in the community. The instrument is designed to measure quality of life in seven domains. (See Attachment A.) The survey will be administered to participants at three points in time—just prior to transition, about 11 months after transition, and about 24 months after transition. The goal of the survey is to provide standard cross-disability performance metrics for people with physical disabilities, cognitive disabilities, intellectual impairments, and/or disabilities due to mental illness. The assessment will enable comparisons across HCBS programs for these transitioning beneficiaries.[23]

  1. The Home and Community-Based Service (HCBS) Experience Survey

CMS is supporting development of a new survey designed to align with the AHRQ Consumer Assessment of Healthcare Providers and Systems (CAHPS) project. CAHPS tools provide a standard benchmark for performance of health care providers, using data obtained from patients and others. CMS is currently testing a new HCBS Experience Survey that can be added to the current group of CAHPS instruments. The goal of the survey is to provide standard performance metrics for HCBS programs that are applicable to all populations served including people with physical disabilities, cognitive disabilities, intellectual impairments, and disabilities due to mental illness. The survey is intended to gather direct feedback from participants in Medicaid HCBS programs, about their experiences with services and supports. The Experience Survey is different from others in that it will provide comparable information on program participants across the spectrum of disability and federally-funded services, regardless of the context or time frame in which participants are receiving HCBS. Survey responses will be compiled to develop quality measures at the program level. The goal for these measures is to enable federal and state governments to expand quality improvement to encompass individual quality of life and outcome measures, across HCBS populations. Data collection for the field test is scheduled to begin in Fall 2012.[24]

6. Standard LTSS/HCBS Quality Measurement Outcome Surveys In Use

  1. Developmental Disabilities National Core Indicators (NCI) Consumer Survey

States administering Medicaid LTSS for people with intellectual and developmental disabilities have long had survey instruments in place that ask the individual recipients their views on the services that they are receiving. Twenty-five states now employ a Developmental Disabilities National Core Indicators Consumer Survey and others are likely to join the effort in the near future. The core indicators are standard measures used across states to assess the outcomes of services provided to individuals and families. Indicators address key areas of concern including employment, rights, service planning, community inclusion, choice, and health and safety.[25][26] (NCI indicators can be accessed at:

  1. Participant Experience Surveys (PES) for HCBS for Elderly and Disabled

In 2003, CMS developed a survey for states to administer to people with disabilities receiving HCBS services. The survey, administered in face-to-face interviews, focuses on: access to care, choice and control, respect/dignity, and community integration/inclusion.[27] AHRQ also developed a survey users’ guide for the CMS survey which includes information about the purpose of the survey; how to select the sample; how to choose and train interviewers; how to schedule and prepare for interviews; general interviewing guidelines; how to code the responses; how to analyze the results; and how to act on the findings.[28] (A copy of the survey can be accessed at:

In one example of a state’s use of the PES, the Texas Department of Aging and Disability Services (DADS) produced a report in January 2011 describing the perceived quality of long-term services and supports administered by DADS, and trends in long-term services and supports over time. Perceived quality and trends over time were obtained by examining responses given by people who receive long-term services and supports to one of two surveys: the National Core Indicators (NCI) survey or the Participant Experience Survey (PES).[29]

Findings suggest that people are satisfied with information about how to access long-term services and supports and receive the services they need. People also reported that their LTSS helped them achieve their personal goals and supported their health and well-being. In addition, findings from people who use the Consumer-Directed Services (CDS) option suggest that people who direct their services and supports have a high degree of awareness about choosing the staff that help them and are more likely than people who do not use CDS to choose the staff that help them.

The report also suggests specific areas for improvement including enhancing opportunities for people to have choice, control, and autonomy over their services and supports. While improving choice, control, and autonomy are broad goals to achieve, the report identified specific opportunities to improve long-term services and supports, including increasing a person’s autonomy to take risks, and having a choice about the staff who help them.[30]

  1. Wisconsin “Personal Experience Outcomes Integrated Interview and Evaluation System (PEONIES)” [31]

Wisconsin has developed tools for measuring outcomes and quality in community based LTSS settings based on “Personal Experience Outcomes.” Such outcome measurement tools seek to account for the wide variety of preferences and expectations that seniors and persons with disabilities may have for the support and assistance they require to live in community settings.

These personal experience outcomes are measured using individual goals:

  • I decide where and with whom I live.
  • I make decisions regarding my supports and services.
  • I decide how I spend my day.
  • I have relationships with family and friends I care about.
  • I do things that are important to me.
  • I am involved in my community.
  • My life is stable.
  • I am respected and treated fairly.
  • I have privacy.
  • I have the best possible health.
  • I feel safe.
  • I am free from abuse and neglect.

Wisconsin asserts that measured outcomes are intended to help care managers and consumers work together to make sure services are supporting the things that are most important to the consumer and to monitor and improve quality. Understanding outcomes also ensures that the programs the state funds are helping people achieve the quality of life they desire.

  1. Personal Outcome Measures developed by the Council on Quality and Leadership (CQL).[32]

CQL is focused on community agencies, and offers products to assist investigators in developing quality measures to determine whether services ensure consumer choice, participant-direction, and individual satisfaction. The systems ask consumers appropriate questions about consumer- choice, participant directed services, and consumer experience and satisfaction. (Please note that CQL provides consultation on person-centered quality measurement; measurement tools are also available for purchase. However, some information is free and downloadable from the organization’s website.)

State Advocates’ Role in Identifying Managed LTSS Outcome Measures

Although no nationally tested and validated managed LTSS outcome measures yet exist, advocates have access to the significant work previously discussed that sets out both important quality domains and constructs, and areas in which information should be collected. Moreover, a number of states are already using some existing managed LTSS outcome surveys that incorporate key concepts required to measure the effectiveness of managed LTSS/HCBS. Until national outcome surveys are available, state advocates should consider quality measurement recommendations from these national research efforts along with existing surveys and related tools.

Core Managed LTSS/HCBS Outcome Measurement Principles

The following core principles are intended to help advocates identify and evaluate key managed LTSS quality measures.

The Measurement Process is Person-Centered

Person-centered means that all aspects of LTSS planning, implementation, and evaluation are directed by the individual with long-term support needs to the maximum extent possible, or by another person important in the life of the individual whom she or he has freely chosen to direct the process. A person-centered approach aims to identify the individual’s strengths, capacities, preferences, needs, and desired outcomes.

An essential element for evaluating the impact of the shift of LTSS to managed care environments will be the measurement of LTSS outcomes from person-centered perspectives including consumer functioning (e.g., availability of support with everyday activities when needed; the presence of friendships; maintenance of family relationships), consumer experience (e.g., respectful treatment by direct service providers; opportunities to make choices about providers; opportunities to make choices about services), and program performance (e.g., receipt of all services in the care plan).

Measurement Examines Quality of Life Outcomes

Measurement should evaluate quality of life outcomes related to the individual’s living situation, choice and control, access to personal care, experience of respect and dignity, extent of community integration, participation, and inclusion, overall life satisfaction, health status, and achievement of person-centered goals. Other measures include but are not limited to the effectiveness of support arrangements, availability of self-directed supports, financial management services including individual budgeting, personal finance and asset building, relationship building and maintenance, education, employment, participation in religious and spiritual activities, and cultural preferences.

Measurement Outcomes Inform Improvement

A measurement and evaluation system for LTSS must incorporate principles of continuous quality improvement. Continuous Quality Improvement (CQI) is an ongoing process measuring and improving individual outcomes and the provision of person-centered LTSS. CQI uses qualitative and quantitative methods to identify needed improvements in both processes and outcomes, implement improvements, and subsequently measure the impact of improvements in LTSS systems and individual quality of life. The basic elements of CQI:

  • System design defines performance measures that will be used to evaluate quality and identify areas for intervention, specifies how data will be collected to monitor program implementation, and embraces quality improvement, including the development of proactive mechanisms to avoid quality problems.
  • Discovery involves the review and analysis of performance measures and other objective data.
  • Remediation is improving overall quality, including both fixing the individual quality problems when they occur, and implementing the systemic changes needed to reach required benchmarks.
  • Improvement implements long-term, system-wide solutions to any quality problems revealed during discovery, and collects and applies data to measure improvement.[33]

Selecting Measures

While it is beyond the scope of this paper to discuss in detail the complex, step-by-step processes typically used to select and adopt measures, the Institute of Medicine (IOM) of the National Academy of Sciences has provided some guidance, including the following selection criteria that may be helpful to advocates as they identify measures that are most important for use in their state.[34]

The IOM groups three criteria according to the subject of measurement:

  • Impact. In the LTSS context, the impact of the provided service, for example, on quality of life, must be considered.
  • Meaningfulness. The measures(s) should be understandable by consumers, advocates, and policymakers and represent concerns and issues that matter to them.
  • Susceptibility. The extent to which measures are influenced by the home and community- based care systems.

The measures should have something to do with aspects of LTSS that various stakeholders, including policymakers, can influence. For example, policymakers should be able to take action on specific problems that are revealed by collected data.

Other IOM criteria pertain to the scientific soundness of the measure:

  • Validity. The measure should have face validity (i.e., it should make sense logically); it should correlate well with other measures of the same aspects of support and care (construct validity) and capture meaningful aspects of such support and care (content validity).
  • Reliability. The measure should produce consistent results when it is used repeatedly and with different groups, especially over time.
  • Feasibility. Feasibility refers, for example, to the practical ability to implement the measure such as availability of measure prototypes, the availability of required data, and the cost or burden to collect the data.

Data Sources

As advocates consider and evaluate various outcome measures, they should be aware of the following data sources that may provide information called for by various measure queries.

  • Program data. Various Medicaid LTSS/HCBS programs have historically been required to collect program data as part of their ongoing operations. For example, provider files, enrollment data, service planning records, care management tracking processes, grievance and complaint data, and audit information are being collected now by Medicaid LTSS/HCBS programs. Advocates should call for such data to be required of managed LTSS/HCBS programs. These data sources then can be drawn upon to fulfill certain outcome measurement requirements.
  • Utilization data. This data relates to services paid for by Medicaid or other government programs and includes, for example, cost-per-member-per-month, hospitalization rates for people with certain conditions or impairments, and frequency with which durable medical equipment is provided.
  • Assessment data. Currently, collection of assessment data is hampered by the fact that states do not use a uniform assessment instrument; therefore it is difficult to compare assessment outcomes. However, under the duals demonstrations and the Balancing Incentives Program, each participating state is required to develop a uniform assessment tool, which will eventually make it possible to collect data using these tools and to measure certain LTSS outcomes across regions of a state or, in some cases, for the entire state.
  • Survey and interview data. Core elements of Medicaid LTSS services such as control, respect, and dignity are frequently measured through surveys. Although surveys collect important information, the data they collect may also present some challenges such as difficulty in scaling the results, linking to other data sources such as utilization and cost data, sample size, and the cost and time associated with data collection and analysis. Nevertheless, surveys can be key methods to identify problem areas that require immediate intervention or improvement.

Generating Reports

Using the data sources previously described, advocates should urge that specific reports be generated that respond to the selected measures. Suggested report topics include:

  • Person centered goal outcome trends against cost and service utilization for each major disability group across the lifespan
  • Beneficiary satisfaction including identification of unmet need
  • Snapshots, changes over time, and long-term trends in number of enrollees in different LTSS settings
  • HCBS and institutional expenditures monthly, quarterly and annually as a percentage of overall LTSS expenditures
  • Average per person expenditures in HCBS and Nursing Facility (NF) settings
  • Average length of residence in HCBS settings
  • Number of new admissions to NFs over 12 months and average length of stay
  • Number of LTSS enrollees transitioned from NFs to HCBS settings over 12 months and, conversely, the number of enrollees admitted to NFs from HCBS settings
  • Summaries and trends on complaints and appeals, especially those related to continuity of care and transition issues

Building Infrastructure Capacity

While collection of outcome data is required to drive overall quality improvement, the capacity of the various relevant entities (including states, MCOs, and local service agencies) must also be adequate in order to ensure effective implementation. Following are capacity-building recommendations related to systems, and to monitoring and reporting.

Incorporation of Person-Centered Focus into Systems

In order to ensure effective implementation and evaluation of the person-centered process, the following specific mechanisms must be in place at the relevant state, MCO and local agency levels:

  • Principles that underlie the person-centered process must be incorporated into relevant policies, mission/vision statements, operations documents, and measurement mechanisms.
  • Staff at all levels, and not just at the front-end direct or customer service level, must have a consistent understanding of person-centered principles, values and implementation processes.
  • Staff and leadership must receive ongoing capacity-building training in the person- centered process.

Monitoring and Reporting Requirements

1. Outreach/Enrollment Monitoring

States and MCOs must monitor outreach and enrollment processes and structures for adequacy of information and referral in the no-wrong-door model, as well as compliance with requirements for adequate enrollment processes. No-wrong-door refers to single entry point systems for accessing LTSS/HCBS that enable consumers to access services through one agency or organization. Undue barriers to gaining access to needed services and supports must be identified and addressed using CQI methods previously described.

  • Any Independent State Ombudsman that oversees the integration of LTSS/HCBS into managed care must have experience and expertise in person-centered LTSS principles, and the capacity to assist with the resolution of both individual and systemic problems revealed by CQI processes.
  • The results of quality measurement surveys and CQI outcomes must be made available to stakeholders and the public in a readily accessible and transparent fashion.

2. Regularly Scheduled Site Visits

States must ensure that MCOs or their designees conduct site reviews regularly in order to gain a qualitative understanding of the environmental context in which data is collected and reported. MCOs must visit a statistically valid random sample of providers in the following categories:

  • Residential settings that are owned by providers must be monitored and measured.
  • Aggregate settings such as day treatment centers, mental health “club houses,” and others must be monitored to ensure that there is adequate community access according to the person-centered plan.
  • A sample of person-centered plans must be reviewed annually in the context of in-person participant interviews for fidelity to the person-centered planning process.

3. Critical Incident Reporting

Critical incident reporting is a well-established quality management mechanism in clinical care and must also be included in LTSS/HCBS quality management. Examples of critical incidents include sexual abuse, suspicious death, physical abuse, neglect, serious injury, frequent care provider absences, financial exploitation, or suicide attempt. Critical incident reports can be used to track both patterns and prevalence of serious adverse events among LTSS beneficiaries and to inform strategies for managing risk, preventing errors, and focusing quality improvement efforts. States and MCOs can parse aggregated incident data in a variety of ways to identify incidents geographically by region or locale, by provider agency, or by type of incident. Such mechanisms for tracking incidents must be part of MCOs’ overall quality management system and they must make data available to states and other interested parties including oversight agencies.[35]

4. Early Warning Reporting

Methods must be developed to ensure that problems that occur before they are identified through the monitoring and reporting processes previously described are resolved proactively. Such problems typically arise when clinical and LTSS/HCBS assessments are siloed, and therefore poorly coordinated. For example, deteriorating wheelchair seating can lead to pressure ulcers, which likely will be treated clinically, yet the underlying problem of inadequate, worn durable medical equipment may be overlooked, along with the negative impact of reduced mobility on quality of life and beneficiary choice and satisfaction.

5. Complaint Reporting

A mechanism for responding to complaints that is free of conflict of interest is an essential component of any health care and HCBS delivery system. Mechanisms for tracking trends in key complaint areas must be a part of the overall quality management system.

Suggested Action Steps

States that are planning to move to managed LTSS must determine what quality measures will be used to identify and evaluate outcomes for individual beneficiaries. Advocates can use the following suggested methods to actively engage with this decision-making process.

  • Request to see any surveys that the state is using currently to measure LTSS beneficiary outcomes for Medicaid HCBS waiver programs[36]
  • Review existing surveys to determine if the data being collected meets the suggested core principles and hews to the domains and related items recommended in research by the national groups cited above
  • Identify areas where additional survey elements are needed
  • Advocate for the state to establish and periodically convene forums to solicit and consider stakeholder and beneficiary input on LTSS quality measures
  • Seek state and MCO-level policies and procedures, and contract language that will ensure an appropriate level of managed LTSS quality measurement
  • Introduce recommended individual-level quality measures to local/regional MCOs planning to take over provision of LTSS

In all of these activities, advocates should recognize both the promise and the limitations of quality measures. Since quality measures rely on data aggregated some time after individual occurrences, quality measures cannot substitute for beneficiary appeal rights and other systemic protections that provide timely remedies for beneficiaries otherwise at risk of bad outcomes.


The trend toward providing Medicaid LTSS through managed care systems is growing rapidly and it is likely that every state will deliver LTSS as a managed service in the near future. Shifting LTSS to MCOs presents some possible opportunities such as enabling expansion of services to more beneficiaries, increasing the types of services that are available, and potentially spurring rebalancing of LTSS funding so that more resources are provided for HCBS. However, advocates also have articulated significant concerns. The widespread lack of experience delivering LTSS on the part of many MCOs, and the speed with which the transition of such services to managed systems is taking place, leave insufficient time for MCOs to prepare adequately to serve a large influx of people with disabilities of all ages. Moreover, on its face, the fundamental structure of managed care raises questions about whether or not the individual needs of beneficiaries will conflict with the cost containment goals and practices of MCOs. For these reasons, advocates, along with consumers who will experience the transition firsthand, must engage with states, MCOs, and providers in order to foster accountability, promote disability literacy, and ensure that the principle of person-centeredness is embedded in each of the key systems of service.

One of the core aspects of engagement will be identification, promotion, adoption and implementation of adequate LTSS outcome measures, which are critical to determining the effectiveness of the new systems in delivering promised services and in protecting the right of people with disabilities of all ages to live safely and with dignity in the community in settings of their choice.


Attachment A
Comparison Chart: LTSS/HCBS Domains and Measures


[1] The Medicaid program allows for the coverage of Long Term Care Services through several vehicles and over a continuum of settings. This includes Institutional Care and Home and Community Based Long Term Services and Supports. In Medicaid coverage, institutional services refers to specific benefits authorized in the Social Security Act. These are hospital services, Intermediate Care Facilities for People with Mental Retardation (ICF/MR), Nursing Facility (NF), Preadmission Screening & Resident Review (PASRR), Inpatient Psychiatric Services for Individuals Under Age 21, and Services for individuals age 65 or older in an institution for mental diseases. Community Based Long-Term Services & Supports aim to provide sustainable, person-driven long-term support systems in which people with disabilities and chronic conditions have choice, control and access to a full array of quality services that assure optimal outcomes, such as independence, health and quality of life. See CMS (Center for Medicare and Medicaid Services), “Long-Term Services and Support,” Information/By-Topics/Long-Term-Services-and-Support/Long-Term-Services-and-Support.html (October 1, 2012).

[2] CMS, “Long-Term Services and Support.”

[3] Paul Saucier, et al., The Growth of Managed Long-Term Services and Supports (MLTSS) Programs: A 2012 Update, prepared for CMS, pp. 6-24, July, 2012, Medicaid-CHIP-Program-Information/By-Topics/Delivery-Systems/Downloads/MLTSSP_White_ paper_combined.pdf (October 1, 2012).

[4] Board on Healthcare Services (HCS), “Current and Future State of Performance Measurement and Reporting,” in Performance Measurement: Accelerating Improvement (Pathways to Quality Health Care Series), 2006, (October 1, 2012).

[5] H. Stephen Kaye, Ph.D, Center for Personal Assistance Services, University of California San Francisco, before the California Senate Human Services Committee, March 27, 2012.

[6] Maureen Booth and Julie Frailich, “Performance Measurement: Managing and Using Home and Community-Based Services Data for Quality Improvement,” Muskie School of Public Service, University of Southern Maine, (October 1, 2012).

[7] National Quality Forum, “Measuring Healthcare Quality for the Dual Eligible Beneficiary Population, Measure Application Partnership, Final Report to HHS,” June, 2012, http://www. Workgroup.aspx (October 1, 2012).

[8] Kaiser Commission on Medicaid and the Uninsured, “People with Disabilities and Medicaid Managed Care: Key Issues to Consider,” February, 2012, upload/8278.pdf. (October 1, 2012).

[9] National Quality Forum, “Measuring Healthcare Quality.”

[10] National Quality Forum, “Measuring Healthcare Quality.”

[11] Agency for Healthcare Research and Quality (AHRQ), Environmental Scan of Measures for Medicaid Title XIX Home and Community-Based Services. AHRQ Publication No. 10-0042-EF, June 2010, (October 1, 2012).

[12] Kaiser Family Foundation, “An Update on CMS’s Capitated Financial Alignment Demonstration Model for Medicare-Medicaid Enrollees,” April 2012, upload/8290.pdf (October 1, 2012).

[13] CMS, “CMS Letter to State Medicaid Directors: Quality Framework for Home and Community Based Services,” February 17, 2004, (October 1, 2012).

[14] National Quality Forum, p. 24. The report also notes that: “Because HCBS are largely non- medical, they necessarily operate within a different quality paradigm than the health system. Many of the primary domains of high-quality, person-centered HCBS can be traced back to the disability rights movement and the historical need to assure adequate quality of life for individuals with disabilities leaving institutional care settings.”

[15] National Quality Forum, p. 25. NQF reviewed a total of 148 potential HCBS measures from three primary sources: Environmental Scan of Measures for Medicaid Title XIX Home and Community–Based Services (June 2010), Raising Expectations: A State Scorecard on LTSS for Older Adults, People with Disabilities, and Family Caregivers (September 2011), and National Balancing Indicator Contractor (October 2010).

[16] It should be noted that measures examining the extent to which beneficiaries obtain appropriate services do not also query whether or not people with disabilities encounter lack of disability awareness and literacy among health care providers, which can affect health outcomes. Measures also do not include architectural and programmatic barriers to care such as lack of accessible diagnostic or exam equipment or needed accommodations such as American Sign Language (ASL) Interpreters, lifting assistance, or additional time for an exam or consultation. Measures examining these issues as they affect health outcomes should be included among any final LTSS/HCBS survey and data collection methods.

[17] National Quality Forum, Exhibit H.

[18] AHRQ, Environmental Scan of Measures.

[19] Heather Young, et al., “Measurement Opportunities and Gaps: Transitional Care Processes and Outcomes Among Adult Recipients of Long-Term Services and Supports,” prepared for Long-Term Quality Alliance (LTQA), December, 2011, (October 1, 2012).

[20] H. Stephen Kaye, Ph.D, testimony, March 27, 2012.

[21] H. Stephen Kaye, Ph.D., “Selected Inventory of Quality-of-Life Measures for Long-Term Services and Supports Participant Experience Surveys,” Center for Personal Assistance Services, University of California San Francisco, (December 2012).

[22] CMS has ongoing initiatives to support and facilitate continuous quality improvement across all HCBS programs. See CMS website Information/By-Topics/Quality-of-Care/Quality-of-Care-HCBS.html. Examples include:

  • HCBS Quality Measurement Projects: Four key quality measurement projects are currently underway in HCBS programs. The projects test a variety of measurement sets that address quality of life, health, satisfaction, impact of program design, and system balancing. Efforts to coordinate the outcomes of those projects are also being considered.
  • Cross-Cutting Quality Efforts in HCBS: HCBS cross-cutting teams work to align quality across HCBS program authorities. Cross-cutting quality goals are focused on advancing quality integration, continuous quality improvement, and information technology.
  • National Technical Assistance: Under the National Quality Improvement initiative, states may request technical assistance from the CMS-designated national TA provider, the National Quality Enterprise (NQE). The NQE provides on-site assistance to states to address specific concerns such as development and implementation of HCBS quality improvement strategies, sampling, evidence gathering, and addressing quality issues that arise in regional office waiver quality reviews.

[23] Matt Sloan and Carol Ivan, “Money Follows the Person Quality of Life Survey,” prepared for CMS, 2007, (October 1, 2012).

[24] “Agency Information Collection Activities: Proposed Collection; Comment Request,” Federal Register 77:97 (May 18, 2012) p. 29644-29645, (October 1, 2012).

[25] National Association of States United for Aging and Disabilities (NASUAD), Annual Summary Report 2009-2010, (October 1, 2012).

[26] National Core Indicators website: (October 1, 2012).

[27] NASUAD, “Participant Experience Survey,” (October 1, 2012).

[28] NASUAD, “Participant Experience Survey.” A copy of the survey users’ guide can be found at

[29] Texas Department of Aging and Disability Services, Long-Term Services and Supports Quality Review 2010 (January 2011): 8.

[30] Texas Department of Aging and Disability Services, Long-Term Services and Supports Quality Review 2010 (January 2011): 9.

[31] See Center for Health Systems Research and Analysis website: (October 1, 2012).

[32] The Council on Quality and Leadership, (November 9, 2012).

[33] Sara Galantowicz, “Implementing Continuous Quality Improvement (CQI) in Medicaid HCBS Programs,” Thomson Reuters, January 2010, webcontent/documents/document/d_007056.pdf. (October 1, 2012).

[34] Committee on the National Quality Report on Health Care Delivery, Envisioning the National Health Care Quality Report, 2001, (October 1, 2012).

[35] Taryn Bowe, et al.,“Managing and Using Home and Community-Based Services Data for Quality Improvement,” Community Living Exchange, Rutgers Center for State Health Policy, p. 20, April 2006, (October 1, 2012).

[36] If the request is denied, advocates can file a formal Freedom of Information Act (FOI) request.

[37] Kane, R. A. (2001). Long-Term Care and a Good Quality of Life: Bringing them closer together. The Gerontologist, 41(3), 293-304.

ATTACHMENT A: Comparison Chart — LTSS/HCBS Domains and Measures

Measure Application Partnership/National Quality Forum Selected Potential Measures for Medicaid Home and Community-Based Services (HCBS) from Three Sources June 2012

1) Framework: HCBS Scan (AHRQ, Thomson Reuters)
DOMAIN Measures/Constructs
Client Functioning
  • Degree to which consumers experience an increased level of functioning.
  • Unmet need in ADLs/IADLs (11 measures total).
  • Degree to which people express satisfaction with relationships.
  • Satisfaction with close friends.
  • Satisfaction with relationships with parents, siblings, and other relatives.
  • Participants reporting unmet need for community involvement.
  • Degree to which people with identified physical health problems obtain appropriate services and degree to which health status is maintained and improved.
Client Experience
  • Degree to which consumers report that staff are sensitive to their cultural, ethnic, or linguistic backgrounds and degree to which consumers felt they were respected by staff.
  • Degree of active consumer participation in decisions concerning their treatment.
  • Case manager helpfulness.
  • Degree to which consumers were satisfied with overall services.
  • Service satisfaction scales: home worker; personal care; home-delivered meals.
Program Performance
  • Ability to identify case manager.
  • Ability to contact case manager.
2) NQF/MAP—Framework: LTSS Scorecard (AARP, The Commonwealth Fund, The SCAN Foundation)
Choice of Setting and Provider Tools and programs to facilitate consumer choice (AARP Scorecard—composite indicator, scale 0-4).
Quality of Life and Quality of Care
  • Percent of adults age 18+ with disabilities in the community usually or always getting needed support.
  • Percent of adults age 18+ with disabilities in the community satisfied or very satisfied with life.
Support for Family Caregivers Percent of caregivers usually or always getting needed support.
3) Framework: National Balancing Indicators (Abt Associates, IMPAQ International)
Sustainability Proportion of Medicaid HCBS spending of the total Medicaid LTCspending.
Self-determination/ Person- centeredness Availability of self-direction options.
Community Integration and Inclusion Waiver waitlist (The waitlist measure may be inappropriate as a measure of community integration and inclusion for states that are dropping wait lists when beneficiaries move to managed care.)
Prevention Proportion of people with disabilities reporting recent preventive health care visits (individual-level).
Coordination and Transparency
  • Proportion of people reporting that service coordinators help them get what they need (individual-level).
  • Coordination between HCBS and institutional services.

Agency for Healthcare Research and Quality (AHRQ) Quality Measures for Medicaid Home and Community-Based Services June 2010

DOMAIN Measures/Constructs
Client Functioning:
  • Change in daily activity function.
  • Availability of support with everyday activities when needed.
  • Presence of friendships.
  • Maintenance of family relationships.
  • Employment status.
  • School attendance (children only).
  • Community integration.
  • Receipt of recommended preventive health care services.
  • Serious reportable adverse health events.
  • Avoidable hospitalizations.
Client Experience:
  • Respectful treatment by direct service providers.
  • Opportunities to make choices about providers.
  • Opportunities to make choices about services.
  • Satisfaction with case management services.
  • Client perception of quality of care.
  • Satisfaction and choice regarding residential setting.
  • Client report of abuse and neglect.
  • Availability of support for resilience and recovery (mental health service recipients only).
Program Performance:
  • Access to case management services.
  • Availability of care coordination.
  • Receipt of all services in the care plan

Center for Personal Assistance Services University of California San Francisco California Senate Human Services Committee March 27, 2012

DOMAIN Measures/Constructs
Quality, adequacy, and impact of services
  1. Basic satisfaction measures related to quality, timeliness, appropriateness
  2. Adequacy of services: Did the person get enough help, or were some of their needs unmet?
  3. Consumer choice, control, direction of services d. Consequences of help received or not received:
    1. Did getting the help enable the person to participate in social, cultural, and/or economic activities?
    2. Did lack of help hinder such participation?
    3. Did problems with help hinder participation, e.g., did person miss appointments, engagements, work, etc., because help did not show up, or did not arrive on time?
  4. Unmet need for services in the population at large, not just among recipients
Health, functional, and healthcare- related outcomes:
  1. Health status including mental health, functional abilities
  2. Injuries or secondary health conditions typically experienced by LTSS recipients, such as falls, burns, skin ulcers, or involuntary weight loss
  3. Maintenance of community living; i.e., avoidance of institutionalization
  4. Healthcare utilization, including avoidable hospitalization, ER visits
  5. Mortality
“Quality of life” and social participation measures: The 11 LTSS-related quality of life domains identified by Rosalie Kane[37] include:

  1. Autonomy/choice
  2. Meaningful activity, which may include employment for working-age adults
  3. Relationships
  4. Individuality
  5. Privacy
  6. Dignity
  7. Sense of safety, security, and order
Family- and family caregiver-focused outcomes
  1. Adequacy of caregiving support services
  2. Caregiving-related emotional stresses
  3. Caregiver physical injuries
  4. Caregiving-related financial stresses
  5. Interface of family caregiving and paid help
Paid personal assistance worker and workforce-related outcomes
  1. Wages, benefits, work hours and conditions, turnover
  2. Training and/or certification
  3. Injuries
  4. Job satisfaction
  5. Local availability of workers to meet consumer demand

Money Follows the Person (MFP) Quality of Life Survey (QoL) (To be tested in 2012)


  • Living situation
  • Choice and control
  • Access to personal care
  • Respect/dignity
  • Community integration/inclusion
  • Overall life satisfaction
  • Health status