Validation Study of Kids Hearing Game: A Self-Administered Pediatric Audiology Application



 

Brian Kungy, MD1; Larisa Kunda, MD1; Sarah Groff, AuD2; Erica Miele, AuD2; Marion Loyd, AuD2; Diane M. Carpenter, MPH3

Perm J 2021;25:20.157

https://doi.org/10.7812/TPP/20.157
E-pub: 03/10/2021

ABSTRACT

Objective: Conduct a comparison study between conventional audiometry and a tablet-based hearing screening application, Kids Hearing Game (KHG). If KHG measures hearing at levels comparable with conventional audiometry, it could be used to screen hearing in children.

Methods: Prospective equivalence study where measurements of pure tone hearing via KHG are compared with measurements of pure tone hearing via conventional audiometry in patients aged 6-11 years over a 4-month period. Eighteen patients completed the study. Results from 36 ears were included in the data for analysis. Decibel measurements from each frequency measured with KHG for each ear were compared with conventional audiometry. Mean measurements were calculated for each ear and frequency as well as mean differences in measurements at each ear and frequency. Tests of equivalence were used to assess mean within-subject differences in decibel measurements using a 10-dB zone of indifference.

Results: Mean decibel measurements using KHG for the right ear at 500, 1000, 2000, 4000, and 8000 Hz and the left ear at 1000, 2000, 4000, and 8000 Hz were found equivalent to conventional audiometry (p < 0.050). The mean decibel measurement using KHG for the left ear at 500 Hz was found not equivalent (p = 0.101). However, when left and right ear data were analyzed together, KHG was found to be equivalent to conventional audiometry across all frequencies. Eight patients having hearing loss greater than 25 dB on conventional audiometry were also identified by KHG to have hearing loss.

Conclusion: KHG is comparable to conventional audiometry and may be used as a screening tool for children.

INTRODUCTION

Hearing loss can have a detrimental effect on childhood development and academic performance. Children with untreated hearing loss may experience mild to moderate learning difficulties, inattention, behavioral problems, and poor social interaction. Children having more severe to profound hearing loss may experience severe learning difficulties, stigmatization, and social isolation.1 Most people are aware of the effects of severe to profound hearing loss in childhood, and often these patients are easily identified. However, the correlation between mild hearing loss and poor academic performance2 along with the aforementioned potential sequelae make screening for hearing loss a very important part of the medical care for children not just in the US but also around the world.

About 11% of school-age children have mild hearing loss.3 It has also come to light that, just as amblyopia exists in childhood visual development, “amblyaudia” can be a lasting consequence of asymmetric hearing loss,4 even in children who have temporary hearing loss from effusions/infection.5 In some of these cases, central connections in the auditory system can be permanently altered through abnormal development, thus leading to permanent potentially detrimental change in the affected ear’s central auditory connections.4 Some researchers have even noted an association between hearing loss and a variety of psychological or psychiatric disorders.6

There is a relative shortage of audiologists (4.1 per 100,000 population, ranging from 7.3/100,000 in Colorado to 1.9/100,000 in California as of 20197) compared with the number of patients who need assessment and treatment, and this shortage is projected to get worse over the next decade.8 After birth, children in the US generally get screening tests at their schools at certain intervals. It is hoped that many will also get an annual screening at their pediatrician’s office. These annual screenings usually consist of some form of pure tone air conduction audiometry through headphones. In the US alone, school screening protocols are variable,9,10 and even in the most proactive districts they may not be done often enough to pick up cases of hearing loss. January 2020 data from the American Speech-Language-Hearing Association states that 8 states (Alabama, Hawaii, Idaho, Iowa, New Mexico, North Dakota, South Dakota, and Wyoming) do not have required school hearing screening, and in 5 states (Missouri, Montana, New Hampshire, South Carolina, and Wisconsin) school hearing screening is suggested but not mandated.11 Generally, children only are referred for formal audiograms if the occasional hearing screening picks up a problem or if the child’s pediatrician or parents believe there is a problem. Because of these deficiencies and the variability in screening, it is likely that many children have undiagnosed hearing loss for several months or longer (in cases of temporary hearing loss) during critical developmental ages. There has been a push to develop a more efficient means of testing children for hearing loss.

With the advent of tablet computing and the huge growth of tablet applications, there is a tremendous opportunity to apply this technology to improve access to hearing screening in children. Children have been found as early as 12 months of age to have at least moderate ability to use a tablet, and in 1 study 90% of subjects age 24 months showed at least moderate ability.12 Although there are already self-administered audiometric software applications on tablets or smartphones that have been shown to adequately measure hearing thresholds in patients,13-15 developers (Hearing.Games, LLC) have designed a new software application, Kids Hearing Game (KHG), which is essentially an audiogram in a game format. Because this application is a game and may appeal to children, we anticipated that a child may be able to maintain focus and concentration to adequately complete the test. The animation, graphics, and interactive gameplay make this application different from other hearing test applications. It should be noted that this application is not meant to replace a formal audiogram or the medical personnel needed to confirm and treat hearing loss; nor will it serve to mitigate inconsistencies and deficiencies in state-mandated school screening. However, it may be useful as a screening tool for concerned and motivated parents to see if a formal audiogram is warranted for their children. We aimed to assess whether a tablet application under conditions similar to real-world conditions performs at a level that is equivalent to audiometry in a clinical setting among pediatric patients belonging to Kaiser Permanente Northern California.

MATERIALS AND METHODS

This study was reviewed and approved by the Institutional Review Board of Kaiser Foundation Research Institute. Pediatric patients ages 6-11 years who were scheduled to have audiograms done for any reason at one Kaiser Permanente Northern California medical center between July 1, 2018 and November 30, 2018 were recruited to participate in the study. Patients were excluded from consideration for the study if they met any of the following criteria: cognitive or developmental dysfunction, non-English speaking, congenital ear malformation, known congenital binaural severe deafness, and disruptive or uncooperative behavior. Written informed consent was obtained from each patient’s parent or guardian, and written assent was obtained from each patient in order to participate in the study. The ear canal was examined before testing to ensure that there was no excessive cerumen or other blockage. Each patient underwent both conventional audiometry in an audiometry booth and the tablet screening application in a quiet room of the office with low ambient noise (< 30 dB). The order of testing was random via coin flip. Subjects were presented with an online gift card of nominal value as a thank-you for enrolling.

Conventional audiometry was performed in a sound-treated booth in the following way. The subject wore headphones, and the audiologist presented tones via air conduction at various frequencies and varying intensities to both the right and left ears. Thresholds were recorded when the subject properly responded to the presented tone 50% of the time. Bone conduction thresholds, speech reception thresholds (via air conduction), speech discrimination thresholds (via air conduction), and tympanometry were performed as part of the standard audiogram. Speech reception thresholds were used to help determine audiogram reliability: if the speech reception thresholds were within 10 dB of the pure tone average, the audiogram was determined to be reliable. For the purposes of the study, only pure tone air conduction thresholds were used for comparison with air conduction thresholds measured by the tablet application.

Tablet application hearing screening was performed in the following way. Amazon Kindle Fire 7 tablets were loaded with the KHG via the Amazon App Store. The subject played the game in a quiet room (not an audiology booth) in the clinic. The game’s sound presentation level was calibrated to Ausdom F01 Wired Over-Ear Headphones. Each subject wore the headphones and played the game on the tablet. The game enacts a first-person view of the player in a boat moving down a river, aiming to catch fireflies. The subject is told to listen carefully for the presented tones and to touch the screen when he or she hears the tone. When the subject responds appropriately, he or she “catches” a firefly. These pure tone sounds are presented via air conduction to the subject at varying frequencies and intensities to the left and right ears. The game sustains the subject’s interest by continuing to provide visual and spoken auditory feedback to the subject during game play. One hundred twenty tones are presented over 5 rounds, or 24 tones per round. There is a built-in mechanism in the game that help to ensure that each subject is properly responding: if the subject touches the screen more than 36 times in 1 round, the subject does not advance to the next round and needs to re-do the round, in which case the series of tones is presented again in a different order. After each subject completes 5 rounds, the game concludes, and the results are recorded. Pictures of gameplay screenshots can be seen in Figures 1 and 2.

tpj20157f1

Figure 1. Screenshot of gameplay - the player has a first person perspective while sitting in a boat moving down a calm river. The yellow lights represent fireflies that are “caught” when a sound is successfully responded to.

tpj20157f2

Figure 2. Screenshot of gameplay taken when a sound is successfully responded to. The stars are seen here, giving that indication. A sound will also accompany this during gameplay.

Data from each ear for each subject were recorded on an Excel spreadsheet. Because the lowest sound threshold level that could be recorded by the tablet was 20 dB, any values under 20 dB recorded via conventional audiometry were entered as 20 dB.

Univariate statistics were used to summarize categorical data as counts and frequencies; for continuous variables, means and standard deviations or medians and interquartile ranges were calculated. The decibel measurement from each frequency (500, 1000, 2000, 4000, and 8000 Hz) measured with the tablet for each ear was compared with corresponding measurements obtained by conventional audiometry within the same subject. Mean measurements were then calculated for each ear and each frequency, as well as mean differences in measurement at each ear and frequency. Tests of equivalence were performed to assess the mean within-subject difference in decibel measurements at 500, 1000, 2000, 4000, and 8000 Hz. The zone of indifference for these comparisons was 10 dB. Analyses were performed using SAS 9.4 (Cary, NC).

RESULTS

During the enrollment period, 19 patients were enrolled and consented, and 18 patients were able to successfully complete both the conventional audiogram and the KHG tablet audiometry application. Data were thus obtained for 36 ears: 18 left ears and 18 right ears. Of the 18 subjects, 12 (66.7%) were male and 6 (33.3%) were female; median age was 7 years (interquartile range: 6-9).

Eight patients were found to have hearing loss with a hearing threshold in any ear measured at ≥ 25 dB at any tested frequency by conventional audiometry. Of these patients, all 8 were found to have at least 1 threshold in any ear measured at 25 dB or above when tested by the KHG tablet application.

Mean dB measurements for each frequency for each ear were measured for both the KHG tablet application and conventional audiometry (Table 1). Based on these measurements, the KHG tablet application was found to be equivalent to conventional audiometry in all frequencies tested (p < 0.050) except for 500 Hz in the left ear (p = 0.101) with a zone of indifference of 10 dB.

Table 1. Equivalence test, comparing tablet to audiometry, by ear (left vs right)

               
Hz Ear Mean value, tablet (± SD) (dB) Mean value, audiometry (± SD) (dB) Difference in means (± SD) (dB) 90% confidence intervala Equivalence p value
500 (n = 18) Left 28.89 (± 11.19) 23.61 (± 8.37) −5.28 (± 15.10) −11.47, 0.91 Not equivalent 0.101
500 (n = 18) Right 27.50 (± 9.89) 22.78 (± 8.26) −4.72 (± 11.56) −9.46, 0.02 Equivalent 0.035
1000 (n = 18) Left 24.72 (± 6.96) 21.39 (± 4.79) −3.33 (± 8.40) −6.78, 0.11 Equivalent 0.002
1000 (n = 18) Right 25.00 (± 5.94) 21.67 (± 4.85) −3.33 (± 6.42) −5.96, −0.70 Equivalent < 0.001
2000 (n = 18) Left 22.22 (± 3.92) 21.67 (± 5.14) −0.55 (± 6.16) −3.08, 1.97 Equivalent < 0.001
2000 (n = 18) Right 22.50 (± 3.93) 22.50 (± 6.48) 0.00 (± 6.64) −2.72, 2.72 Equivalent < 0.001
4000 (n = 18) Left 23.06 (± 6.67) 20.56 (± 2.36) −2.50 (± 7.33) −6.14, 1.14 Equivalent < 0.001
4000 (n = 18) Right 25.28 (± 8.13) 21.94 (± 5.18) −3.33 (± 8.57) −6.85, 0.18 Equivalent 0.002
8000 (n = 18) Left 24.17 (± 6.47) 21.11 (± 2.74) −3.06 (± 7.50) −6.13, 0.02 Equivalent < 0.001
8000 (n = 18) Right 25.28 (± 8.82) 21.67 (± 5.14) −3.61 (± 6.82) −6.41, 0.82 Equivalent < 0.001

a. The zone of indifference for the test of equivalence was ± 10 decibels (dB). To establish equivalence, the 90% confidence interval needs to fall within the limits of (−10.00, 10.00).

When left ear and right ear data were combined, the mean dB measurements for the KHG tablet application and conventional audiometry were found to be equivalent at all frequencies (p < 0.050, Table 2). The difference between the means was particularly small at 2000 Hz at 0.28 dB, whereas the difference was larger at 500 Hz at 5.00 dB. The mean dB measurements for the tablet were higher than for conventional audiometry across all frequencies.

Table 2. Equivalence test, comparing tablet to audiometry, both ears (left and right combined)

               
Hz Ear Mean value, tablet (± SD) (dB) Mean value, audiometry (± SD) (dB) Difference in means (± SD) (dB) 90% confidence intervala Equivalence p value
500 (n = 36) Both 28.19 (± 10.43) 23.19 (± 8.21) −5.00 (± 13.26) −8.73, −1.27 Equivalent 0.015
1000 (n = 36) Both 24.86 (± 6.38) 21.53 (± 4.75) −3.33 (± 7.37) −5.41, −1.26 Equivalent < 0.001
2000 (n = 36) Both 22.36 (± 3.87) 22.08 (± 5.78) −0.28 (± 6.32) −2.06, 1.50 Equivalent < 0.001
4000 (n = 36) Both 24.17 (± 7.42) 21.26 (± 4.03) −2.92 (± 7.87) −5.13, −0.70 Equivalent < 0.001
8000 (n = 36) Both 24.72 (± 7.65) 21.39 (± 4.07) −3.33 (± 7.07) −5.32, −1.34 Equivalent < 0.001

a. The zone of indifference for the test of equivalence was ± 10 decibels (dB). To establish equivalence, the 90% confidence interval needs to fall within the limits of (−10.00, 10.00).

Of all enrolled subjects, 6 were referred by pediatrics subsequent to failed hearing screenings. Thirteen were referred by the otolaryngology department. Of all subjects, regardless of referring department, 7 subjects had pressure equalizing tubes or previously extruded pressure equalizing tubes (1 with a tympanic membrane perforation from a tube), 1 subject had bilateral retractions, 3 had recent findings of serous otitis media, and 8 were found to have normal tympanic membranes.

DISCUSSION

This equivalence study comparing hearing measurements taken with the KHG tablet audiometry application and conventional audiometry shows that equivalence was largely achieved. This shows that KHG could be a useful screening tool for children thought to have some degree of hearing loss.

There are potential variables that may make KHG suboptimal for screening outside of the doctor’s office and can lead to inaccuracies. First, the KHG tablet application was run using ordinary headphones (under $20) in a relatively quiet room (under 30 dB ambient noise), whereas conventional audiometry uses more sophisticated headphones, which are calibrated on a routine and regular basis, in a sound-treated audiometry booth. There is more potential for some competing noise to make it more difficult for the subject to hear the tones being presented on the tablet application. It is also possible that in real-life use outside of the clinic an environment quiet enough to test with the tablet would not be able to be achieved; this could affect accuracy. Second, although the calibration procedure in KHG has been optimized to adjust the volume of the tablet based on the type of tablet and the type of headphones, there could be inconsistencies that could affect the accuracy of sound measurements in real-life use. Finally, children having any of the conditions for which we excluded (cognitive or developmental dysfunction, non-English speaking, congenital ear malformation, binaural severe deafness, and behavioral problems) would likely have a suboptimal experience playing this game.

Although our study was adequately powered to show statistical equivalence between mean thresholds measured, a larger number of patients studied could further validate these findings. A study was performed by the developers of the application using 516 pediatric subjects in China.16 It demonstrated 85% agreement of KHG hearing measurements with conventional audiometry at 500, 1000, and 2000 Hz with decreased levels of agreement at 4000 Hz and 8000 Hz.16 It also showed a sensitivity of 91% in detecting hearing loss 20 dB or greater and a specificity of 74% in subjects age 4 years or older, with the specificity increasing to 90% in their subjects age 7 years or older.16 The language discrepancy with the Chinese-speaking subjects playing the English-only KHG application may have contributed to some decreased levels of correlation of hearing measurements in that study.16 There were also remarks from subjects in that study that the gameplay took too long and could become tedious, leading to potentially less accurate results.16 We also noticed similar sentiments from several subjects in this study. There was 1 patient in our study (a potential subject who was excluded from the final cohort) who turned the tablet off twice during gameplay. Results from that patient were not able to be recorded.

Another potential limitation to tablet and phone-based audiometry applications is that they require an internet connection and compatible hardware/equipment. Therefore, it may only be available to motivated, at least somewhat technologically sophisticated, and financially stable families at this time; this would be even more of a problem outside of developed countries. However, as access to technology continues to improve and education about the existence of these applications becomes more widespread, these barriers should become reduced over time.

With some modifications to KHG, including changes to significantly shorten gameplay without compromising the accuracy of testing (which may help younger subjects with shorter attention spans) and the development of versions in different languages, we believe it could become more accurate with different patient populations. Larger studies than ours, using subjects who primarily communicate in the same language as the language used in KHG, should be conducted to confirm our findings.

CONCLUSION

Pediatric hearing loss, whether mild to severe, progressive, permanent, or even temporary, can lead to lasting detrimental developmental and academic consequences. Tablet screening audiometry applications such as KHG can be used to test and screen for hearing loss in children who may not have adequate access to resources for traditional hearing screening, let alone conventional audiometry. These applications may also be used by concerned family members to screen their children for hearing loss without needing to go to the pediatrician’s or audiologist’s office. This may be particularly beneficial during the years when traditional school hearing screening is not offered or in cases in which school screening is not available. Although these screening tools do not replace conventional audiometry, our results suggest that they may help to detect hearing loss in children. This may allow affected children to be identified earlier and may allow them to undergo conventional audiometry earlier than they otherwise would have to confirm the presence of hearing loss. If screening tools such as KHG allowed hearing loss to be detected and treated earlier, they could be of tremendous benefit to neurocognitive development, social development, and academic performance in these patients.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgments

Funding for this study was provided by the KPNC Community Benefit program.

Author Affiliations

1Department of Head and Neck Surgery,Kaiser Permanente, Union City, CA

2Department of Audiology, Kaiser Permanente, Union City, CA

3Division of Research, Kaiser Permanente, Oakland, CA

Present address: Brian Kung, MD, Department of Otolaryngology, Kaiser Permanente, 201, 16th Ave E, Seattle, WA 98112.

Corresponding Author

Brian Kung, MD ()

Author Contributions

Brian Kung, MD: study design, data analysis, manuscript preparation. Larisa Kunda, MD: study design, data analysis, manuscript preparation. Sarah Groff, AuD: data collection, manuscript preparation. Erica Miele, AuD: data collection, manuscript preparation. Marion Loyd, AuD: data collection, manuscript preparation. Diane Carpenter, MPH: study design, data analysis, manuscript preparation.

Funding Statement

Funding for this study was provided by the Kaiser Permanente Northern California Community Benefit program.

Abbreviations

dB = decibel; Hz = hertz; KHG = Kids Hearing Game; SAS = Statistical Analysis System

References

1. Olusanya BO, Newton VE. Global burden of childhood hearing impairment and disease control priorities for developing countries. Lancet 2007 Apr;369(9569):1314-7. DOI: https://doi.org/10.1016/s0140-6736(07)60602-3

2. Daud M, Noor R, Rahman N, Sidek D, Mohamad A. The effect of mild hearing loss on academic performance in primary school children. Int J Pediatr Otorhinolaryngol 2010 Jan;74:67-70. DOI: https://doi.org/10.1016/j.ijporl.2009.10.013

3. Bess FH, Dodd-Murphy J, Parker RA. Children with minimal sensorineural hearing loss: Prevalence, educational performance, and functional status. Ear Hear 1998 Oct;19(5):339-54. DOI: https://doi.org/10.1097/00003446-199810000-00001

4. Kaplan A, Kozin E, Remenschneider A, et al Amblyaudia: Review of pathophysiology, clinical presentation, and treatment of a new diagnosis. Otolaryngol Head Neck Surg 2016 Feb;154(2):247-55. DOI: https://doi.org/10.1177/0194599815615871, PMID:26556464

5. Uclés P, Alonso MF, Aznar E, Lapresta C. The importance of right otitis media in childhood language disorders. Int J Otolaryngol 2012;2012:818927. DOI: https://doi.org/10.1155/2012/818927

6. Carvill S. Sensory impairments, intellectual disability and psychiatry. J Intellect Disabil Res 2001 Dec;45(Pt 6):467-83. DOI: https://doi.org/10.1046/j.1365-2788.2001.00366.x, PMID:11737534

7. Brook G. Annual workforce data: 2019 ASHA-certified audiologist- and speech-language pathologist-to-population ratios; 2019. https://www.asha.org/siteassets/uploadedfiles/2019-Audiologist-and-SLP-to-Population-Ratios.pdf

8. Windmill I, Freeman BA. Demand for audiology services: 30-yr projections and impact on academic programs. J Am Acad Audiol 2013 May;24(5):407-16. DOI: https://doi.org/10.3766/jaaa.24.5.7, PMID:23739060

9. American Speech-Language-Hearing Association. State licensure trends and quarterly updates. Accessed March 28, 2017. http://www.asha.org/advocacy/state/StateLicensureTrends/.

10. Sehkar DL, Zalewski TR, Paul IM. Variability of state school-based hearing screening protocols in the United States. J Community Health 2013 Jun;38:569-74. DOI: https://doi.org/10.1007/s10900-013-9652-6

11. American Speech-Language-Hearing Association. School-Age Hearing Screening. Accessed January 16, 2020. https://www.asha.org/Advocacy/state/School-Age-Hearing-Screening/.

12. Hourcade JP, MascherSL, WuD, PantojaL. Look, my baby is using an iPad! an analysis of YouTube videos of infants and toddlers using tablets. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, Seoul, Republic of Korea; 2015, pp 1915-24.

13. Yeung J, Javidnia H, Heley S, Beauregard Y, Champagne S, Bromwich M. The new age of play audiometry: Prospective validation testing of an iPad-based play audiometer. J Otolaryngol Head Neck Surg 2013 Mar;42:21. DOI: https://doi.org/10.1186/1916-0216-42-21

14. Yeung J, Heley S, Beauregard Y, Champagne S, Bromwich M. Self-administered hearing loss screening using an interactive, tablet play audiometer with ear bud headphones. Int J Pediatr Otorhinolaryngol 2015 Aug;79(8):1248-52. DOI: https://doi.org/10.1016/j.ijporl.2015.05.021, PMID:26055197

15. Whitton J, Hancock K, Shannon J, Polley D. Validation of a self-administered audiometry application: An equivalence study. Laryngoscope 2016 Oct;126(10):2382-8. DOI: https://doi.org/10.1002/lary.25988, PMID:27140227

16. Xiao L, Zou B, Gao L, et al. A novel tablet-based approach for hearing screening of the pediatric population, 516-patient study. Laryngoscope 2020 Sep;130:2245-51. DOI: https://doi.org/10.1002/lary.28329.

Keywords: amblyaudia, audiology, hearing loss, hearing screening, pediatrics

ETOC

Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.

CIRCULATION

2 million page views of TPJ articles in PubMed from a broad international readership.

Indexing

Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, CrossRef, and SciVerse/Scopus.


                                             

 

 

ISSN 1552-5775 Copyright © 2020 thepermanentejournal.org

All Rights Reserved