To Spread or Not to Spread: That is the Question …That Evaluation Can Answer


Jim Bellows, PhD; Ana Jackson, PhD; Courtnee Hamity, PhD;
Margaret Wang, PhD; Lunarosa Peralta, MPH;
Care Management Institute, Center for Evaluation & Analytics

Introduction: Program evaluation informs decisions at each stage of Kaiser Permanente’s (KP’s) Accelerating Learning and Spread (XLS) model. The objective of this minicourse was to guide participants in using the right evaluation tools and methods at the right time to optimize KP programs and make evidence-informed decisions about scale and spread.
Methods: The Care Management Institute’s Evaluation team developed and shared a framework for Evaluation for XLS, informed by KP Colorado’s framework in a National Academy of Medicine discussion paper. The framework includes the “5 Rights” of program evaluation: Right People, Right Questions, Right Design, Right Data, and Right Analysis and Interpretation. We also designed an Evaluation for XLS Roadmap workbook and SharePoint site to guide participants in program evaluation planning and execution throughout a program’s life cycle.
Results: Using case studies involving recent KP Voh’s Awards winners, participants, ranging from improvement advisors, evaluators, researchers, analysts, program champions, and sponsors, worked in inter-role groups to apply the roadmap to common evaluation challenges. After course completion, participants were given the evaluation roadmap to continue strengthening their developmental or impact-evaluation efforts.
Discussion: Participants learned how to identify questions, methods, tools, and resources to help inform program optimization and decision making. Moving forward, courses that align evaluation methods with the XLS model will contribute to augmenting evaluation capacity at KP by ensuring widespread access to evaluation resources, consultation, and peer review within the organization. All minicourse materials can be found at: and Evaluation.aspx.


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021

All Rights Reserved