Telehealth in Pediatric Gastroenterology Can Be a Sustainable Long-Term Option: A Single-Center Experience


Kristina Leinwand, DO1,2; Nicole Blodgett, RN1; Ramya Ramraj, MD1,2

Perm J 2021;25:20.191 [Full Citation]
E-pub: 01/06/2021


Telehealth has been a well-established system of care delivery at Kaiser Permanente Northwest Pediatric Gastroenterology since before the COVID-19 era, where 54% of our completed pediatric gastroenterology encounters in the past year were virtual visits. Although it was previously limited in its use across the majority of health care in the US, telehealth is now a widely accepted system in the face of the COVID-19 pandemic. However, this unprecedented rapid expansion of telehealth within both new and established telehealth systems is not without challenges or quality concerns. In this report, we present our single-center experience from the past year where virtual care was already an integral part of our clinical practice in pediatric gastroenterology and review satisfaction data and quality measures from the COVID-19 era where our existing telehealth system expanded to nearly exclusive virtual care.


Telehealth has been recognized as an efficient, cost-effective, and high-quality method of clinical care delivery.1,2 Many health care organizations have explored various forms of telehealth including electronic consultations (e-consults), tele-education, tele-monitoring, or virtual encounters, as progressive means to increase timely access to specialty providers, such as gastroenterologists.3,4 However, implementation of telehealth has been limited due to challenges in medical infrastructure, licensing, reimbursement, and fear of litigation regarding safety and quality of care.1,5,6 Clinician acceptance has also been one of the factors in the slow integration of virtual care.7

Use of telehealth in pediatric gastroenterology and most other pediatric subspecialties has been limited. However, telehealth has now become increasingly relevant as practices have shifted to providing nearly exclusive virtual care in the midst of the COVID-19 pandemic.8,9 This inimitable situation has required expansion of formerly established telehealth systems and quickly forced protocol development to overcome the limitations of virtual care that were observed prior to the pandemic.10 Although pediatric gastroenterologists have been actively trying to come up with protocols10 and share their telehealth experiences during the pandemic,11,12 much uncertainty still exists regarding the safety, efficacy, and quality of care provided in a nearly exclusive telehealth system.

Kaiser Permanente Northwest (KPNW) Health Care is an integrated health care delivery system that serves about 622,000 members in the states of Oregon and Washington, of which 155,000 members are 0 to 18 years of age. The KPNW health care system has been practicing a robust telehealth system for several years, where virtual encounters have become increasingly popular in both primary and specialty care. Improved access to care, efficiency, convenience, and flexibility are some of the preferred reasons expressed by families for using telehealth. The proportion of virtual care grew as the virtual visits and telehealth models were reported to deliver care better than initially anticipated.13 Over the past 3 years, the percentage of virtual visits in KPNW pediatric gastroenterology, which consists of two full-time physicians, has significantly increased and comprised 54% of the completed visits in the past year.

In this report, we present our single-center telehealth experience from the past year where virtual care has been an integral part of our clinical practice in pediatric gastroenterology and review the satisfaction and quality measures during the COVID-19 era.

Telehealth Encounters Pre-COVID

All referrals are placed electronically within the KPNW electronic medical record (EMR) and reviewed and triaged directly by a pediatric gastroenterologist. In this process (Figure 1A and 1B), growth chart patterns, labs, imaging, and prior clinic visit notes and correspondence with relevant providers are extensively reviewed. After chart review, the referral is processed as either e-consult or appointed for a visit. Recommendations for e-consults are sent via EMR documentation to the referring providers. These recommendations remain visible and attached to the original chart referral as a reference to all providers within the health care system who may provide care for the patient. The majority of new referrals are offered face-to-face (F2F) appointments. Select patients are offered primary virtual appointments based on diagnosis, travel distance to clinic, and other social factors in order to improve access to care. For follow-up visits, families are offered a choice of F2F or virtual visits (via video or telephone) if deemed appropriate for virtual follow-up by the provider. A F2F follow-up visit was more commonly assigned for certain diagnoses, such as inflammatory bowel disease or chronic liver disease, or when multidisciplinary care (MDC) was needed. If families chose video visits, instructions were sent in advance on how to check in to the video appointment using the Kaiser Permanente app.

tpj20191f1 copy

Figure 1. Referral triage process. (A) A small percentage of new patients were appointed for first encounter virtually depending on reason for referral or to expedite care. (B) The majority of new patients were seen in office for the first time.

Some of the salient features of the process are:

1. Thorough review of the chart: One of the lessons learned early in the process is that a thorough review of the chart is essential to telehealth practices. The comprehensive review of the recent chart notes from the referring provider including review of physical examination, vital signs, growth and weight parameters, labs, and imaging studies affords the reviewing provider both competence and confidence in their medical decision-making process, triage of visit, and in determining appropriateness for virtual visit.

2. Clear pathway for communication: The importance of concise, direct communication with families about the benefits of a virtual visit, such as earlier appointing times and the ability to expedite the evaluation process, was established very early in the process of expanding telehealth. Families were also reassured that an office visit can be completed subsequently if desired or as medically necessary. Often, when a perceived delay in F2F visit contributed to anxiety for any new patients and their families, a preliminary virtual visit helped address urgent concerns and start evaluation or management when appropriate. For e-consults, a clear method for documenting recommendations in the EMR and communicating them directly to the referring provider were paramount to the success of the referral and triage process.

3. Patient examinations: Patient examinations completed by the referring provider at the time of referral are reviewed in detail by the pediatric gastroenterologist. Although not intended as a replacement for the expert specialty examination, reviewing the physical examination performed and documented by referring providers improved triage and e-consult competency. For the select new referrals that were determined appropriate for proceeding directly to endoscopic evaluation after the initial virtual visit (such diagnoses as isolated dysphagia or elevated tissue transglutaminase IgA antibody), a complete physical examination was performed on the day of endoscopy.

4. Recall system: To complement our telehealth system, a meticulous reminder/recall system is maintained electronically in the EMR. Patient recalls are established for phone visits with our nurses for symptom updates, check in during infusion visits, reviews of expected weight checks, or reminder calls to start medications or to obtain labs. Medical assistants also rely on the EMR recall system to schedule follow-up appointments. Our pediatric gastroenterology MDC providers, including dieticians, social workers, and behavioral health therapists, use the recall system for high-quality patient care.

At KPNW Pediatric Gastroenterology from March 2019 to Feb 2020, 12.5% of new patient encounters, 69% of follow-up encounters, and 54% of all encounters overall were completed using telehealth (Table 1). The most common referral diagnoses for new patients directly appointed to a telehealth visit included elevated transglutaminase IgA antibody, isolated dysphagia, and isolated reflux (Table 2). Occasionally, urgent referrals for diagnoses such as neonatal cholestasis or high suspicion for inflammatory bowel disease were also appointed as an initial virtual visit in order to expedite the necessary workup and provide timely care. Less than 5% of referrals received in 2019 and 2020 were reviewed exclusively as e-consults.

Table 1. Total number and type of new and follow-up visits seen in the pediatric GI clinic in the past 1 year (March 2019-February 2020, pre-COVID)

  Total visits Total new Total follow-up
  1846 473 1368
F2F 838 414 424
Virtual 1003 59 944
% Virtual 54 12.5 69

A total of 59 new patients (12.5%) and a total of 944 follow-up patients (69%) were seen virtually with telehealth, with an overall virtual visit total of 54%.

F2F = face-to-face.

Table 2. A total of 59 new patients were seen virtually from March 2019 to February 2020, classified by specific diagnosis or reason for referral

Virtual new visits n (%)
Elevated tissue transglutaminase IgA antibodies 26 (44)
Dysphagia 8 (14)
GERD 6 (10)
Diarrhea 4 (7)
Abdominal pain 4 (7)
Rectal bleeding 2 (3)
Nausea/vomiting 2 (3)
Familial adenomatous polyposis 2 (3)
Miscellaneous 5 (8)

GERD = gastroesophageal reflux disease.

Quality Measures of Telehealth in the COVID-19 Era

During the initial few weeks of the pandemic from March to May 2020, there was rapid expansion of telehealth, and clinical care became exclusively virtual. Both new and follow-up visits were 100% virtual. In order to quickly assess quality of care and patient satisfaction, a telephone survey was conducted among the patients and families who were seen virtually during the initial weeks. Patients and families were asked to rate the quality of a virtual visit and to compare the experience to a F2F visit.

A total of 45 patient/family satisfaction surveys were completed. Overall quality of the visit was ranked on a scale from 1 to 10, with a score of 1 representing the worst experience and a score of 10 representing a superior experience. The mean rating for the overall quality of visit was 8.9. When asked if the virtual visit experience was the same, better, or worse than the experience of a F2F visit, 78% of patients/families rated the experience as either the same as or better than a F2F visit, whereas only 22% rated the experience worse than a F2F visit (Table 3). The most common reasons families cited for virtual care being a superior experience were convenience and not needing to drive. The reasons virtual care was considered inferior by families were the lack of lack of interpersonal experience and technical difficulties.

Table 3. Patient/family surveys rating quality of virtual care visit and comparison of experience to face-to-face visits (N = 45)

Quality of visit rating (1-10) Mean 8.9 (±1.3)
Virtual care vs F2F experience  
 Same 58%
 Worse 22%
 Better 20%

A total of 45 surveys were completed, with mean quality of visit at 8.9 (±1.3) out of 10, and 78% of patient/families ranked their virtual visit as the same or better experience than face-to-face (F2F) visits.

As an early assessment of quality and safety and to gather information on the reliability, reproducibility, and ability to meet standard of care within a now nearly exclusive telehealth system, intraprovider concordance for e-consult and new encounter recommendations was tested. Twenty e-consults and 15 new patient virtual encounters were evaluated separately by the 2 pediatric gastroenterologists within KPNW to determine intraprovider reliability. Provider 1 reviewed e-consults and virtual visits that were previously completed by Provider 2, and vice versa. E-consult chart review data were restricted to data within the chart at the time of receipt of the referral, and new patient virtual visit data were restricted to the virtual encounter documentation, growth data, labs, and imaging available at the time the virtual visit occurred in real time. The reviewing provider was blinded to the assessment and plan recommended to the family, and rates of concordance between the original recommendations from Provider 1 and the independent recommendations from Provider 2 were measured. The rates of agreement for intraprovider e-consult and new patient telehealth recommendations were 95% and 93%, respectively.

Limitations of Telehealth

The COVID-19 pandemic has forced significant change in the way health care is practiced across the world. Some of the essential changes implemented to our preexisting telehealth system during the pandemic included appointing all initial referrals as virtual visits, implementing virtual MDC care in the same way as it was done in the F2F setting, as well as consolidation of F2F clinics when our patients absolutely need to be seen in the office for their care. Within this single-center experience, one of the more dissatisfying moments of telehealth as perceived by the authors is with virtual care for functional gastrointestinal disorders; family dynamics, social stressors, and other nonmedical contributors to symptoms may not be properly gleaned or assessed without the direct observations of a F2F visit. Another significant limitation for virtual visits exists with providing patient education for a new diagnosis of a chronic disease; anecdotally, the education provided in a F2F visit was perceived to be superior by the provider to when conducted virtually, although no difference has been reported by the families receiving the education.

A limitation in the scenario of exclusive virtual care is the potential absence of recent physical examination data for e-consultation or for medical decision-making in virtual visits. Our pre-COVID-19 robust telehealth system at KPNW has provided high quality, safe care for our patients within the infrastructure of an integrated system that relied on the routine physical examinations by referring providers, but this nearly exclusive virtual care system lacking examination data has not been rigorously tested. Limitations in our quality and satisfaction data include the small number of reviews conducted, the method of intraprovider review without investigating interprovider reviews for reliability and standard of care measurements, and the subjective nature of satisfaction surveys.

Future prospective studies are needed to gather comprehensive, objective data on the safety, efficacy, and quality within a nearly exclusive telehealth care system such as what we are currently experiencing during the COVID-19 pandemic. Close scrutiny will need to be applied to investigating the short-term and long-term complications and disease outcomes. It will be also of interest to compare the pre- and post-COVID-19 encounter patient satisfaction data.


Within the KPNW health care system, there is a focus on providing affordable, high-quality health care within an integrated system. This model allowed for a methodical, unified approach to implementing virtual visits and telehealth slowly over the past several years. Now more than ever, telehealth system expansion is increasingly relevant; the unprecedented national development of nearly exclusive virtual care during the COVID-19 pandemic will drive further investigations into the once limiting factors of telehealth, including patient satisfaction, safety, and quality of care delivery.14 It is clear that even after the COVID-19 pandemic is over, the current practice of almost exclusive virtual care is going to significantly affect how pediatric gastroenterology is delivered on a global level. More families may prefer the ease, affordability, and flexibility of virtual care. Health care providers will recognize the crucial ways in which telehealth expansion will provide superior access to quality care for remote communities and how this earlier access to care can decrease disease complication rates and improve health outcomes. Insurance companies will need to recognize the increasing advantages of virtual care and not refuse payments or provide subpar reimbursements. As the scope of virtual care vastly increases from the pre-COVID-19 era, further innovations will be needed to allow for the seamless integration of telehealth into care delivery systems for pediatric gastroenterologists and their delivery of timely and quality health care.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Author Affiliations

1Pediatric Gastroenterology, Kaiser Permanente Northwest, Portland, OR

2Division of Pediatric Gastroenterology, Doernbecher Children’s Hospital, Oregon Health & Science University, Portland, OR

Corresponding Author

Ramya Ramraj, MD (

Author Contributions

Kristina Leinwand, DO, participated in the conception and design of the manuscript, analysis and interpretation of data, drafting and critical revisions of the manuscript for important intellectual content, and final approval of the version to be published. Nicole Blodgett, RN, participated in acquisition, analysis, and interpretation of data; drafting and critical revisions of the manuscript for important intellectual content; and final approval of the version to be published. Ramya Ramraj, MD, participated in conception and design of the manuscript, analysis and interpretation of data, drafting and critical revisions of the manuscript for important intellectual content, and final approval of the version to be published. All authors are accountable for all aspects of the work.

Source of Funding

None declared.

How to Cite this Article

Leinwand K, Blodgett N, Ramraj R. Telehealth in pediatric gastroenterology can be a sustainable long-term option: A single-center experience. Perm J 2021;25:20.191. DOI:


1. Siegel CA. Transforming gastroenterology care with telemedicine. Gastroenterology 2017 Apr;152(5):958-63. DOI:, PMID:28192101

2. Smith AC, Scuffham P, Wootton R. The costs and potential savings of a novel telepaediatric service in Queensland. BMC Health Serv Res 2007 Mar;7:35. DOI:, PMID:17331259

3. Keely E, Canning S, Saloojee N, Afkham A, Liddy C. Improving access to gastroenterologist using eConsultation: A way to potentially shorten wait times. J Can Assoc Gastroenterol 2018 Sep;1(3):124-8. DOI:, PMID:31294353

4. Stone JM, Gibbons TE. Telemedicine in pediatric gastroenterology: An overview of utility. Telemed J E Health 2018 Aug;24(8):577-81. DOI:, PMID:29271722

5. Liddy C, Drosinis P, Keely E. Electronic consultation systems: Worldwide prevalence and their impact on patient care-a systematic review. Fam Pract 2016 Jun; 33(3):274-85.DOI:, PMID:27075028

6. Bradford NK, Caffery LJ, Smith AC. Telehealth services in rural and remote Australia: A systematic review of models of care and factors influencing success and sustainability. Rural Remote Health 2016 Oct-Dec;16(4):3808. PMID:27744708.

7. Wade VA, Eliott JA, Hiller JE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res 2014 May;24(5):682-94. DOI:, PMID:24685708

8. Finkelstein JB, Nelson CP, Estrada CR. Ramping up telemedicine in pediatric urology: Tips for using a new modality. J Pediatr Urol 2020 Jun;16(3):288. Epub ahead of print 2020 Apr 17DOI:

9. Brough HA, Kalayci O, Sediva A, et al. Managing childhood allergies and immunodeficiencies during respiratory virus epidemics - the 2020 COVID‐19 pandemic: A statement from the EAACI‐section on pediatrics. Pediatr Allergy Immunol 2020 Jul;31(5):442. DOI:

10. Wosik J, Fudim M, Cameron B, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc 2020 Jun;27(6):957. DOI:

11. Berg EA, Picoraro JA, Miller SD, et al. COVID-19-a guide to rapid implementation of telehealth services: A playbook for the pediatric gastroenterologist. J Pediatr Gastroenterol Nutr 2020 Jun;70(6):734-40.DOI:, PMID:32443021

12. Verstraete SG, Sola AM, Ali SA. Telemedicine for pediatric inflammatory bowel disease in the era of COVID-19. J Pediatr Gastroenterol Nutr 2020 Jun;70(6):e140. DOI:

13. Barkholz D. Kaiser Permanente chief says members are flocking to virtual visits. Modern healthcare website. Accessed April 20, 2020.

14. Smith AC, Thomas E, Snoswell CL, et al. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020 Jun;26(5):309-13. DOI:, PMID:32196391

Keywords: coronavirus, pandemic, telemedicine, virtual visits


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021

All Rights Reserved