To Spread or Not to Spread: That is the Question …That Evaluation Can Answer

To Spread or Not to Spread: That is the  Question …That Evaluation Can Answer


Jim Bellows, PhD; Ana Jackson, PhD; Courtnee Hamity, PhD;
Margaret Wang, PhD; Lunarosa Peralta, MPH;
Care Management Institute, Center for Evaluation & Analytics

Introduction: Program evaluation informs decisions at each stage of Kaiser Permanente’s (KP’s) Accelerating Learning and Spread (XLS) model. The objective of this minicourse was to guide participants in using the right evaluation tools and methods at the right time to optimize KP programs and make evidence-informed decisions about scale and spread.
Methods: The Care Management Institute’s Evaluation team developed and shared a framework for Evaluation for XLS, informed by KP Colorado’s framework in a National Academy of Medicine discussion paper. The framework includes the “5 Rights” of program evaluation: Right People, Right Questions, Right Design, Right Data, and Right Analysis and Interpretation. We also designed an Evaluation for XLS Roadmap workbook and SharePoint site to guide participants in program evaluation planning and execution throughout a program’s life cycle.
Results: Using case studies involving recent KP Voh’s Awards winners, participants, ranging from improvement advisors, evaluators, researchers, analysts, program champions, and sponsors, worked in inter-role groups to apply the roadmap to common evaluation challenges. After course completion, participants were given the evaluation roadmap to continue strengthening their developmental or impact-evaluation efforts.
Discussion: Participants learned how to identify questions, methods, tools, and resources to help inform program optimization and decision making. Moving forward, courses that align evaluation methods with the XLS model will contribute to augmenting evaluation capacity at KP by ensuring widespread access to evaluation resources, consultation, and peer review within the organization. All minicourse materials can be found at: and Evaluation.aspx.

The Permanente Journal

Sponsored by the eight Permanente Medical Groups, The Permanente Journal advances knowledge in scientific research, clinical medicine, and innovative health care delivery.

Reprint Permissions

The Permanente Journal welcomes requests for reprints and reproduction. Use of any and all material published in The Permanente Journal is copyrighted and protected.

The Permanente Press

The Permanente Press publishes The Permanente Journal and books related to healthcare. Journal subscriptions are entered for the calendar year. Advance payment in US dollars is required.

ISSN 1552-5775 Copyright © 2018

The Permanente Press. All Rights Reserved.