Challenges Developing Measurement Tools

Common sense would dictate that a person should want to purchase quality when choosing any product or service, and as health care costs soar in the United States, we also want to ensure that we, as consumers of health care and taxpayers who subsidize health care, are reaping maximum quality for that cost (Buck, Godfrey, & Morgan, 1996). According to McGlynn (1997), the costs for health care in the U.S. have been rising dramatically causing disruption in the manner of which professionals provide care and patients seek it out. It is important to realize the impact that these increasing costs and other changes have on the delivery of care, and, as McGlynn points out, assessment of quality measures are the means of evaluation. Unfortunately, McGlynn and others at the time have found quality measures to be lacking the requisite data needed to make an accurate evaluation of the delivery of health care (Brook, McGlynn, & Shekelle, 2000; Grimshaw & Russell, 1993; McGlynn, 1997).

Over the past decade, many efforts have been made to develop quality measures in order to direct quality improvement; however, these efforts, though effective, have been disjointed and ad hoc at best. McGlynn and Asch (1998) cautions that careful attention to methodology is essential when developing these measures. Accurate methodologies can be reproduced and used to effectively compare efforts between institutions. This leads to a best practices continuum of health care provision.

Recently, researchers have studied teamwork behaviors and their influence on patient and staff-related outcomes, but many of the discussions were institution-centric and may not have applied in the macro environment of U.S. health care. Reader, Flin, Mearns, and Cuthbertson (2009) recently attempted to organize these studies and develop a portable and robust framework which would lead to the development of effective team performance and provide means of further testing and improvement of team dynamics. Their findings suggest that effective teamwork is crucial to providing patient care in critical settings. Reader et al. shows one of the shortcomings of recent quality measure development but also illustrates a manner in which to overcome the limitations.

Developing methods for measuring and evaluating performance in health care have been challenging, overall. Campbell, Braspenning, Hutchinson, and Marshall (2002) identify three component issues to addressing these challenges: “(1) which stakeholder perspective(s) are the indicators intended to reflect; (2) what aspects of health care are being measured; and (3) what evidence is available?” (p. 358). This addresses the qualitative concerns of capturing indicators, while efforts like those of Steyerberg et al. (2010) concern themselves with quantitative abstraction and portability, as well as predictive value. Steyerberg et al. promotes the use of reclassification, discrimination, and calibration when using statistical models to develop valid prediction models and novel performance measures.

Performance indicators that are an accurate reflection of health care provision can lead to development of best practices, lower overall health care costs, and improve the delivery of care which will decrease mortality and morbidity. When considering these performance indicators, especially during development, researchers and administrators need to ensure the validity of the measurements. Approaches to developing quality improvement measures are constantly evolving, and new and novel methods are being designed to standardize the instruments, the application, and the reporting. Quality improvement is still, however, a challenge to many health care providers.


Brook, R. H., McGlynn, E. A., & Shekelle, P. G. (2000). Defining and measuring quality of care: a perspective from US researchers. International Journal of Quality in Health Care, 12(4), 281–95. doi:10.1093/intqhc/12.4.281

Buck, D., Godfrey, C., & Morgan, A. (1996). Performance indicators and health promotion targets (Discussion paper No. 150). York, UK: Centre for Health Economics, University of York. Retrieved from

Campbell, S. M., Braspenning, J., Hutchinson, A., & Marshall, M. (2002). Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care, 11(4), 358–364. doi:10.1136/qhc.11.4.358

Grimshaw, J. M. & Russell, I. T. (1993). Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet, 342(8883), 1317-1322. doi:10.1016/0140-6736(93)92244-N

McGlynn, E. A. (1997). Six challenges in measuring the quality of health care.Health Affairs, 16(3), 7-21. doi:10.1377/hlthaff.16.3.7

McGlynn, E. A. & Asch, S. M. (1998). Developing a clinical performance measure. American Journal of Preventive Medicine, 14(3), Supp. 1, 14–21. doi:10.1016/S0749-3797(97)00032-9

Reader, T. W., Flin, R., Mearns, K., & Cuthbertson, B. H. (2009). Developing a team performance framework for the intensive care unit. Critical Care Medicine, 37(5), 1787-1793. doi:10.1097/CCM.0b013e31819f0451

Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., … Kattane, M. W. (2010). Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 21(1), 128–138. doi:10.1097/EDE.0b013e3181c30fb2