OJPHI: Vol. 5
Journal Information
Journal ID (publisher-id): OJPHI
ISSN: 1947-2579
Publisher: University of Illinois at Chicago Library
Article Information
©2013 the author(s)
open-access: This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
Electronic publication date: Day: 4 Month: 4 Year: 2013
collection publication date: Year: 2013
Volume: 5E-location ID: e21
Publisher Id: ojphi-05-21

Detecting Changes in Chief Complaint Word Count: Effects on Syndromic Surveillance
Jessica Sell*
Robert Mathes
Marc Paladini
NYC Department of Health and Mental Hygiene, Long Island City, NY, USA
*Jessica Sell, E-mail: jsell@health.nyc.gov

Abstract
Objective

To identify changes in emergency department (ED) syndromic surveillance data by analyzing trends in chief complaint (CC) word count; to compare these changes to coding changes reported by EDs; and to examine how these changes might affect the ability of syndromic surveillance systems to identify syndromes in a consistent manner.

Introduction

The New York City (NYC) Department of Health and Mental Hygiene (DOHMH) receives daily ED data from 49 of NYC’s 52 hospitals, representing approximately 95% of ED visits citywide. Chief complaint (CC) is categorized into syndrome groupings using text recognition of symptom key-words and phrases. Hospitals are not required to notify the DOHMH of any changes to procedures or health information systems (HIS). Previous work noticed that CC word count varied over time within and among EDs. The variations seen in CC word count may affect the quality and type of data received by the DOHMH, thereby affecting the ability to detect syndrome visits consistently.

Methods

The daily mean number of words in the chief complaint field were examined by hospital from 2008–2011. Spectral analyses were performed on daily CC word count by hospital to explore temporal trends. Change Point Analysis (CPA) using Taylor’s Method with a maximum change level of four was conducted on the CC field by hospital using 1,000 bootstrap samples. According to Taylor, a level one change is the most important change detected on the program’s first pass through the data. For this analysis, a change point was considered significant if it was level 1, detected an average change of more than 0.50 words per day, and was sustained for at least 6 months before a level 2 change of at least 0.50 words occurred. Results of the CPA were compared to reported changes identified by a survey conducted by DOHMH staff of 49 hospitals that collected information about their HIS and coding practices, including any recent system changes.

When a significant level one change was identified, time series graphs for six months before and after the change were created for five syndromes (cold, diarrhea, fever-flu, influenza-like-illness, and respiratory) and the syndrome’s constituent symptom categories (e.g. cough fever, etc.). Changes in syndrome count and composition at the level one change in word count were noted.

Results

The mean chief complaint word count across all hospitals from 2008 – 2011 in NYC was 3.14, with a range of 0 to 18 words. CPA detected a significant level 1 change in 21 hospitals, with a mean change of 0.60 words, with 9 increases (mean= 0.71 words) and 12 decreases (mean= 0.53 words). According to the results of a survey of 49 NYC EDs, 19 have changed coding practices or Health Information Systems since 2008. CPA identified a coincident and significant shift in word count for 8 of these hospitals. CPA also detected significant shifts in word count for 13 hospitals that did not report any changes. Figure 1 shows the results of CPA from one ED in NYC

We observed immediate changes in daily syndrome count after the detected change in CC word count. For example, respiratory syndrome count increased with increased word count and decreased with decreased word count for 10 of the 21 EDs with a significant change in word count. Only 2 EDs saw an opposite effect on respiratory syndrome count. Meanwhile, 9 EDs saw no obvious change in respiratory syndrome count. Furthermore, these changes in daily CC word count coincided with subsequent changes in syndrome composition, the breakdown of syndromes into constituent symptoms.

Conclusions

Change Point Analysis may be a useful method for prospectively detecting shifts in CC word count, which might represent a change in ED practices. In some instances changes to CC word count had an apparent effect on both syndrome capture and syndrome composition. Further studies are required to determine how often these changes happen and how they may affect the quality of syndromic surveillance.

ojphi-05-21f1.tif
[Figure ID: f1-ojphi-05-21]
Figure 1: 

Mean daily CC word count with CPA results marked.



References
Taylor WA. 2000“Change-Point Analysis: A Powerful Tool for Detecting Changes” Retrieved July 5, 2012, from http://www.varia-tion.com/cpa/tech/changepoint.html

Article Categories:
  • ISDS 2012 Conference Abstracts

Keywords: Chief Complaint, Word Count, Change Point Analysis.




Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org