Composite Evaluation Guidance 

Access the Full Report 

The Opportunity

Healthcare is a complex and multidimensional activity, and measurement of its quality should reflect that complexity. Individual measures can provide important information; however, there is tremendous value in summarizing performance by combining information from multiple measures. Composite measures that combine multiple individual measures into one single measure have become increasingly common in healthcare performance measurement and public accountability applications.

The 2006 Institute of Medicine (IOM) report, Performance Measurement: Accelerating Improvement noted that composite measures can enhance measurement to extend beyond tracking performance on separate measures and can provide a potentially deeper view of the reliability of the care system¹, in addition to supporting efforts to rate providers by the quality of their care. Since clinical quality for a given area (e.g., AMI care) can be assessed with multiple indicators, a composite can provide a more comprehensive picture of care quality for multiple stakeholders, including consumers and purchasers.

Composite measures are complex and require a strong conceptual and methodological foundation with different considerations for testing and analysis, and as with individual measures, measure construction affects the reliability, validity, and usefulness of the composite measure. Several composite measures comprise NQF’s portfolio of endorsed measures, and NQF has previously developed guidance to assist steering committees with their assessment of these measures as part of the NQF evaluation process. However, approaches for creating composite measures vary, and can lead to differing performance scores across methods.² 

About the Project

Given the rapid evolution of the field, increasing interest in, and varied types of composite measures, this project will reassess NQF’s existing guidance for evaluating composites, with particular consideration of recent changes in composite measure development and related methodology. The emerging methodological issues that will be addressed include:

• clarifying and distinguishing between the types of composite measures such as all-or-none composites, for which the components are assessed at the patient level; or composites for which the performance measure scores for the healthcare entity are combined in some way (such as a weighted average);
• identifying the advantages and disadvantages of all/none versus weighted composite methodologies;
• identifying testing and analysis needed to demonstrate reliability and validity; and
• determining when component measures should be NQF endorsed (e.g., when individual performance measures scores are combined in a composite vs. components of all-or-none composites) and implications for measure harmonization. 

Process 

Using NQF’s existing composite measure evaluation guidance and recent developments in composite measure development and methodology, NQF will convene a technical expert panel (TEP) through an in-person meeting and several conference calls/webinars to:

• review the existing guidance for evaluating composite measures within the context of NQF’s measure evaluation criteria;
• review existing methods of creating composites and implications on reliability and validity.
• identify a set of prioritized methodological issues that can guide composite measure development and implementation; and
• provide additional recommendations for evaluating future composite measures.

This work will culminate in a guidance report that will identify additional methodological and other significant considerations for NQF’s composite measure evaluation guidance. The report will be available for Public and Member comment and for the Consensus Standards Approval Committee (CSAC) and Board of Directors’ input and approval.

NQF Related Work 

• Composite Measure Evaluation Framework and National Voluntary Consensus Standards for Mortality and Safety - Composite Measures    

Funding 

This project is funded under a contract provided by the Department of Health and Human Services. 

For information about the availability of auxiliary aids and services for NQF’s federally funded projects, please visit: http://www.medicare.gov/about-us/nondiscrimination/nondiscrimination-notice.html.

Contact Information 

For further information, contact Karen Johnson or Elisa Munthali at 202-783-1300 or via email at compositeframework@qualityforum.org. 

Notes 

1. Institute of Medicine, Performance Measurement: Accelerating Improvement, Washington, DC: National Academies Press; 2006
2. Reeves et al. Combining multiple indicators of clinical quality: An evaluation of different approaches. Medical Care 45, no. 6. June 2007.