Patient-Centered Research from Electronic Medical Records

Kaiser Permanente, The Permanente Medical Group, The Permanente Federation


Mikel Aickin, PhD

Fall 2011 - Volume 15 Number 4

Many important scientific discoveries were made not by minor modifications of previous research, but by rethinking fundamentals. At the turn of the past century, Albert Einstein (and a number of others) considered what would happen if we discarded our age-old idea that time and space were absolute, and the result was a revolution in physics. There are many examples in medicine where the previous world-view was overthrown along the way to the germ theory of disease. Charles Darwin struggled with the doctrine that species were created suddenly at one point in time, and arrived at one of the most powerful ideas in the world of biology.

At this time many areas of biomedical research seem to be stalled, or at least only slowly progressing. Despite the National Heart, Blood, and Lung Institute's two-generation campaign to reduce salt consumption in the American population, our best research seems to be ambiguous, if not discouraging, about the extent to which sodium restriction would prevent cardiovascular disease and death. Although some claim that obesity is a national epidemic with serious consequences for a range of diseases, and ultimately death, others say that the results of lowering weight are scarcely measurable at the individual level, and that the truly effective preventive strategies lie elsewhere. One can go on to list conditions for which the consensus view about effective treatment is that there is no consensus view or effective treatment.

Under Researched

Perhaps the current largest collection of unresolved or poorly resolved medical issues have to do with the kinds of patients that primary care physicians see most often. For example, one can find articles over the past 30 years complaining about the basic lack of progress in the treatment of the patient with chronic pain, and the situation does not appear to be improving. As the Institute of Medicine has recently observed,1 there is no National Institute of Pain, so why should we expect to see a comprehensive national research policy in that area? But another reason for not doing clinical research on pain is that the causes are often obscure, the choice of effective therapies is cloudy, and patients with chronic pain tend to bring with them comorbidity profiles that greatly complicate research designs. Although pain may be the primary example, there are many others.

It is a strange situation that many of the routine problems that annoy clinicians are under-researched—the paradox that the most prevalent conditions are also the research orphans. One obvious explanation is that it is so difficult to do randomized clinical trials for these conditions, aggravated by the increasingly evident fact that such trials suffer from a lack of generalizability to clinical populations. In casting around for a research alternative, the obvious thing to try is medical research from existing electronic medical records (EMR). Surely there is a treasure-trove of useful data locked up in EMR systems, just waiting to be liberated.

Electronic Medical Record

It would, however, be equally true to say that there are virtually unlimited supplies of fresh water locked up in the polar ice caps, just waiting to be liberated. The trick is how to do it. Conventional wisdom in biomedical research is that one must rely on randomized clinical trials for the best evidence in any area of medicine. The practical result of this belief is that the efforts even to try to use EMR-based data have been few and far between. Funding has been virtually nonexistent, but the suspicion is that researchers likely see no benefit in preparing proposals for EMR-based research.

In a recent literature review of EMR-based studies,2 it was found that nearly all were published in specialty journals, and for the most part the designs were attempts to import the methods of randomized clinical trials into the EMR data setting. From the titles one would judge that almost all of these studies were narrowly conceived, being concerned with a special slice of the patient population, or a highly specific point on some nuance of a disease process. On the one hand, there are good reasons to doubt that this path (mimicking clinical trials) will be successful, but on the other hand, a substantial number of new techniques for dealing with nonintervention designs has been developed.3

To return to the idea that introduced this essay, the conventional belief is that randomized clinical trials are the primary way to progress in treating disease. Virtually all of our statistical methods are oriented toward this approach. Methods for dealing with nonintervention dilemmas exist but are disused, and we are not making progress in employing them to solve large problems in our delivery of health care. The fundamental that we need to rethink is whether, unlike the polar ice caps, the information in EMRs is not beyond our grasp, and that all we need to do is to reach.

Even if funding were available, and researchers were willing to try it, there is still a large problem doing EMR-based research: finding a place to publish the results. Nearly all biomedical journals expect, or often demand, that articles be in a format that is uniquely tailored for clinical trials. Moreover, they expect conclusions be framed using recognized statistical methods that have been ritualized over decades of debate. It is not unreasonable to anticipate that EMR-based research submitted to existing journals would probably not survive to the review stage.

For this reason, the editors of The Permanente Journal (TPJ) have undertaken an experiment to lower the barrier to publication of EMR-based research to develop the quality and scope of this nontraditional approach. The editors recognize that EMR-based research has features that require modifications in the usual publication format. Chief among these is a succinct report in the usual paper-based form in the Journal itself, backed up by a far less restrictive electronic form, including aspects that are too difficult or expensive to print­—for example, large databases and complex graphics.

In 2012, TPJ will open a new section dedicated to the publication of research that makes fundamental use of medical records data to generate knowledge about therapeutic outcomes. Submissions to this section will be peer-reviewed like all other submissions, but articles that are accepted will appear only in the TPJ's open-access e-journal­—the official journal of record (—with notices in the print journal.

In addition to the reasons noted above for this addition to TPJ, intervention research is too expensive to shoulder the entire burden of evidence-based medicine. There are simply too many conditions and types of patients for us to be able to rely on the clinical trial as the main source of therapeutic knowledge. Even if we could afford to do all the trials that need to be done it is not always obvious that a therapy administered in the setting of a trial is the same as would be administered in usual care. Yet there is still a need to assess and compare therapies based on observation of the individual patient.

Electronic Medical Record Information

There is an immense store of information in existing EMRs, and this storage is planned to increase in size over the coming years. In the past, EMR systems have been used for practice management, to facilitate patient care, and to improve the quality of health care delivery. They have not, however, been used very often for therapeutic medical research. There is a real barrier to this usage—EMR systems have always been designed for uses other than research. This means that some data that would be required for the purposes of high-quality research is either not collected or not stored. It does not mean, however, that the data collected cannot contribute anything to therapeutic research.

The national clinical research enterprise has concentrated its time and money so completely in the intervention-based clinical trial, that EMR-based research has been largely ignored. This has had two consequences. Methods of analysis that deal with the biases caused by lack of a research intervention have not been developed to the same degree as methods for intervention trials. With a perceived lack of methodology, and an actual lack of funding, nonintervention research has not produced the successful studies that form the basis for a larger share of the national research agenda. Thus the information locked away in EMR systems remains there, because of lack of adequate tools for its extraction, and inadequate funding to do the work.

Electronic Medical Record Article Characteristics

The intent of the section is to encourage articles with the following characteristics:
•    fundamental use of data from EMRs
•    assessment of therapies as they are actually provided
•    inclusion of all relevant patients, without narrow exclusion criteria
•    development of analytical methods that address the weaknesses of nonintervention studies
•    portrayal of the complexity of clinical science.

These criteria are not intended to be definitive. Other areas of appropriate research have to do with (in no particular order):
•    description of patient populations, from presenting characteristics through outcomes
•    longitudinal studies of trajectories of care and outcome
•    methods of extracting or recoding data for research
•    qualitative research from EMR narratives
•    potential appearance of long-term adverse consequences associated with various therapies
•    methods for visualizing patterns in complex administrative data
•    changes in the use of therapies over time
•    involvement of medical students and residents in EMR-based clinical science
•    inadequacies of existing EMR systems for research purposes, with recommendations for future changes
•    comparisons of EMR-based research with intervention-based research
•    studies based on non-EMR systems.

Topics of lesser interest for this particular section include:
•    health care service delivery
•    physician adherence to practice guidelines
•    application of intervention trial methods to EMR data
•    study of therapies that would not be used in practice (such as placebos)
•    therapeutic effectiveness assessed without treatment comparisons
•    epidemiologic studies.

The articles in this new section will appear on the Internet to eliminate restrictions on manuscript length and on numbers of tables and (colored) figures, and to open the possibility of including data files, computer programs, or other nontraditional materials. The intent is to provide enough space to deal with the complexity of clinic-based research without arbitrary technical limits.

This initiative is consistent with several recent recommendations regarding the national system of clinical research. For example, an Agency for Healthcare Research and Quality (AHRQ) study4 has promoted the creation of patient registries for specific conditions, which would be primarily based on EMR.

The Institute of Medicine has visualized health record data as a vastly underutilized information resource, and set the goal of an expanded learning health care system substantially based on clinical records.5 Prokosh and Ganslandt have provided a careful outline of the challenges that follow from these recommendations along with some potential solutions.6

1.    Institute of Medicine. Relieving pain in America: A blueprint for transforming prevention, care, education, and research. Washington DC: The National Academies Press, 2011.
2.    Dean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ. Review: Use of electronic medical records for health outcomes research: a literature review. Med Care Res Rev 2009;66(6):611-38.
3.    Guo S, Fraser MW. Propensity score analysis: statistical methods and applications. Thousand Oaks, CA: Sage Publications, 2010.
4.    Gliklich RE, Dreyer NA, eds. Registries for evaluating patient outcomes: a user's guide. 2nd ed. (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc d/b/a Outcome] under Contract No.HHSA29020050035I TO3.) AHRQ Publication No.10-EHC049. Rockville, MD: Agency for Healthcare Research and Quality. 2010 Sep.
5.    Institute of Medicine. Clinical data as the basic staple of health learning: Creating and protecting a public good: workshop summary. Washington, DC: The National Academies Press, 2010.
6.    Prokosch HU, Ganslandt T. Perspectives for medical informatics. Reusing the electronic medical record for clinical research. Methods Inf Med;2009;48(1):38-44.

Celebrating 20 years (copy)


The Winter 2017 issue of The Permanente Journal marked the first issue of our 20th anniversary year. We've grown a lot in the past 20 years, currently reaching an online readership of more than one million worldwide. We've expanded to publish books and a literary and visual arts e-journal, leaflet. We are indexed in MEDLINE and continue to grow our digital footprint. We look forward to continuing to bring you more high-quality content during the next 20 years.


25,000 print readers per quarter, 7,628 eTOC readers, and in 2016, 1.4 million page views on TPJ articles in PubMed from a broad international readership.


The Permanente Journal (ISSN 1552-5767) is published quarterly by The Permanente Press. Journal subscriptions are entered for the calendar year. Advance payment in US dollars is required.


Articles, editorials, letters to the editor, and other material represent the opinion of the authors. Send your comments to

Copyright 2017 The Permanente Journal - Kaiser Permanente. All Rights Reserved.