To Spread or Not to Spread: That is the Question …That Evaluation Can Answer

To Spread or Not to Spread: That is the  Question …That Evaluation Can Answer


Jim Bellows, PhD; Ana Jackson, PhD; Courtnee Hamity, PhD;
Margaret Wang, PhD; Lunarosa Peralta, MPH;
Care Management Institute, Center for Evaluation & Analytics

Introduction: Program evaluation informs decisions at each stage of Kaiser Permanente’s (KP’s) Accelerating Learning and Spread (XLS) model. The objective of this minicourse was to guide participants in using the right evaluation tools and methods at the right time to optimize KP programs and make evidence-informed decisions about scale and spread.
Methods: The Care Management Institute’s Evaluation team developed and shared a framework for Evaluation for XLS, informed by KP Colorado’s framework in a National Academy of Medicine discussion paper. The framework includes the “5 Rights” of program evaluation: Right People, Right Questions, Right Design, Right Data, and Right Analysis and Interpretation. We also designed an Evaluation for XLS Roadmap workbook and SharePoint site to guide participants in program evaluation planning and execution throughout a program’s life cycle.
Results: Using case studies involving recent KP Voh’s Awards winners, participants, ranging from improvement advisors, evaluators, researchers, analysts, program champions, and sponsors, worked in inter-role groups to apply the roadmap to common evaluation challenges. After course completion, participants were given the evaluation roadmap to continue strengthening their developmental or impact-evaluation efforts.
Discussion: Participants learned how to identify questions, methods, tools, and resources to help inform program optimization and decision making. Moving forward, courses that align evaluation methods with the XLS model will contribute to augmenting evaluation capacity at KP by ensuring widespread access to evaluation resources, consultation, and peer review within the organization. All minicourse materials can be found at: and Evaluation.aspx.

etoc emailClick here to join the eTOC list or text TPJ to 22828. You will receive an Email notice with the Table of Contents of each issue.

The Permanente Journal advances knowledge in scientific research, clinical medicine and innovative health care delivery. It is a peer-reviewed journal of medical science, social science in medicine, and medical humanities.

The Permanente Press

The Permanente Press publishes The Permanente Journal and books related to health care. For information about subscriptions, missing issues, billing, subscription renewal, and back issues, Email:


27,000 print readers per quarter, 15,350 eTOC readers, and in 2018, 2 million page views of TPJ articles in PubMed from a broad international readership.


The Kaiser Permanente National CME Program designates this journal-based CME activity for 4 AMA PRA Category 1 Credits. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

ISSN 1552-5767 Copyright © 2019

All Rights Reserved.