OJPHI: Vol. 4 Issue 2:
Journal Information
Journal ID (publisher-id): OJPHI
ISSN: 1947-2579
Publisher: University of Illinois at Chicago Library
Article Information
©2012 the author(s)
open-access: This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
collection publication date: Year: 2012
Electronic publication date: Day: 14 Month: 9 Year: 2012
Volume: 4 Issue: 2
E-location ID: ojphi.v4i2.4198
DOI: 10.5210/ojphi.v4i2.4198
Publisher Id: ojphi-04-17

Quality and integration of public health information systems: A systematic review focused on immunization and vital records systems
Joshua R Vest1
Hilary M Kirk2
L Michele Issel2
1Weill Cornell Medical College, New York, NY
2University of Illinois at Chicago, School of Public Health, Chicago, IL
Correspondence: Corresponding Author: Joshua R Vest, PhD, MPH, Assistant Professor, Weill Cornell Medical College, Email: Jov2025@med.cornell.edu

Abstract
Objectives:

Public health professionals rely on quantitative data for the daily practice of public health as well as organizational decision making and planning. However, several factors work against effective data sharing among public health agencies in the US. This review characterizes the reported barriers and enablers of effective use of public health IS from an informatics perspective.

Methods:

A systematic review of the English language literature for 2005 to 2011 followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) format. The review focused on immunization information systems (IIS) and vital records information systems (VRIS). Systems were described according to the structural aspects of IS integration and data quality.

Results:

Articles describing IIS documented issues pertaining to the distribution of the system, the autonomy of the data providers, the heterogeneous nature of information sharing as well as the quality of the data. Articles describing VRIS were focused much more heavily on data quality, particularly whether or not the data were free from errors.

Conclusions:

For state and local practitioners to effectively utilize data, public health IS will have to overcome the challenges posed by a large number of autonomous data providers utilizing a variety of technologies.


Introduction

Both local health departments (LHD) and state health agencies need access to quantitative information for organizational decision making, strategic and community planning, as well as for the day-to-day practice of public health.1 For example, data exchanged by public health practitioners on childhood immunizations are not only used to determine if specific individuals are up-to-date on vaccinations, but as a core public health indicator, are essential for population health surveillance and for conducting quality improvement processes. Integration of their information systems (IS) is a prerequisite to having access to and sharing of real-time data across organizations with different data management systems and different data elements.

Unfortunately, major forces have conspired to prevent easy and effective data sharing in public health from being the norm. First, public health IS very much a reflection of the public health system in the US: a federation of independent states with differing relationships with their respective LHDs, tasked with various aspects of data collection, storage, and dissemination.2,3 Second, the utilized information technologies (IT) and IS in local health departments may not stem from public health needs and requirements, but instead from a local government’s broader needs, resources and existing IT/IS decisions.4 Furthermore, last decade’s relatively high level of preparedness funds awarded to public health, while providing a much needed upgrade to the IT/IS capacities of state and local agencies, has not been sustained and pre-dated the more recent advances and focus on interoperability seen in the broader health IT arena. Lastly, public health informatics is a relatively new specialty within public health and still has uneven uptake across public health agencies.5 All in all, public health organizations historically have not implemented IS or adopted IT that support efforts at efficient and effective storage and sharing of data.4,6,7

The objective of this literature review is to characterize the barriers and enablers of effective use of public health IS that support two key public health activities: immunizations and vital statistics. We purposefully selected these immunization information systems (IIS) and vital records information systems (VRIS) because they reflect longstanding key public health activities, and both have undergone recent, widespread moves toward complete electronic record keeping.8,9 Furthermore, these systems reflect different types of data, data sources, organizational involvement, and uses within the public health system. With a particular focus on supporting individual vaccinations, a comprehensive IIS includes information from public health organizations, primary care providers and even schools. In contrast, VRIS requires cooperation predominantly from local hospitals, funeral homes, and midwives, to support legal documents for individuals. The results of the literature review provide insights into the means to improve the fragmented state of public health data in the US.

Framework

This review examines the factors related to successful or unsuccessful information sharing from the dual perspectives of IS content and overall structure. Content is critical as that is the aspect of the system that is most visible practitioners relying on the IS for their daily public health work, whereas the structure of the IS in terms of data sources and technology affects the availability and quality of data in a less obvious manner for end users. We used the AIMQ’s dimensions of data quality identify and categorize attributes of the data contained within IIS and VRIS.10,11 The fifteen data quality dimensions addressed such issues as completeness, timeliness, relevancy and understandability. The structural aspect of IS intergration is aptly captured at the system-level by Hasselbring’s dimensions of IS integration: heterogeneity, distribution, and autonomy.12 The myriad of technological factors that can plague large information systems that have diverse sources of data and multiple types of organizational users fall under the heterogeneity dimension.

Problematically, barriers to interoperability originating from a lack of data and exchange standards, difficulties in networking legacy systems, and different technology platforms are commonplace in public health. For example, LHDs differ substantially in their methods of record keeping,13 and for some public health activities no national data standards exist. The distribution dimension captures the scope of the IS in terms of the number of data sources, such as local and state government agencies and private healthcare providers. The autonomy dimension includes characteristics of the organizations exchanging information as those characteristics relate to interorganizational relationships, self-determination, and governance issues. As autonomous entities, organizations and individuals may exhibit idiosyncratic behaviors around actual system usage, willingness to engage in interorganizational data sharing, or even approaches to providing public health activities.


Methods

We undertook a systematic review to identify the factors associated with successful and unsuccessful IIS and VRIS following the suggested form of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).14

Information sources & searching

We conducted a review of the English language public health and medical research literature for research, evaluation, or case studies describing experiences with the implementation, evaluation, success, usage, or failure of immunization information systems and vital records systems in the US. Searches were limited to articles published between 2005 and 2011. Both immunization information and vital record systems were searched as keyword terms in PubMed, ISI Web of Science, and the Center for Public Health Systems & Services Research Library. In addition, we reviewed the table of contents for the same time period in relevant journals, organization’s websites and conference abstracts (a sample query and all sources are provided in the Appendix). Initial search results yielded 756 unduplicated records.

Study selection

Based on abstract information, we excluded from the initial search set all non-US based studies, reviews, editorials, commentaries, and those articles that did not describe an immunization information or vital statistics system or efforts to create such, or instances where no indication existed the study was about an IS. Two members of the research team independently reviewed each record and then arrived at the excluded set through consensus. The same team members independently read the full text of each article and determined its inclusion status. Differences were resolved by consensus after a joint reading session.

Articles were retained for inclusion in the review if it described the barriers to, or factors supporting, the design, implementation, or effective use of an IIS or VRIS used by state, local or government public health agencies. These criteria allowed for the inclusion of studies addressing technical structure, data quality, or evaluations of the IS as used in practice. This excluded special purpose surveys, IS maintained solely by private providers, systems that provided only aggregated information, or articles that described the source data as part of a large research endeavor (i.e. using mortality data to supplement a hospital research study).

Data collection

From each included article, we abstracted the type of IS, study design, goals and objectives, study setting or participants, and factors identified as contributing or hindering success of the effort. We then coded each of the identified factors according to the dimensions of IS integration or data quality.


Results
Study selection & characteristics

We screened 756 unduplicated records and of those 210 warranted full text review (Figure 1). The most common reason for excluding records after full text review was that the article did not identify any barriers to, or factors supporting, either any aspect of the IS or data quality. Our review process resulted in 23 total studies of which 11 reported on immunization information system (IIS),1,1524 11 on vital record systems,2535 and 1 described a public health information system that integrated vital records with other information systems.36

Sixteen articles described state-level systems, but reports also included data and systems specific to Atlanta,25 Boston,20 New York city,15,21 and Philadelphia.18,19 One study interviewed immunization program managers both in urban areas and at the state level.1 In addition, the majority of included studies (17 out of 23) were cross-sectional analysis of the information system records, often compared to data generated in clinical settings or in other information systems.1621,2328,3035 The remaining articles were case studies,15,22,36 a review of documents,29 and qualitative.??1

ojphi-04-17f1.tif
[Figure ID: f1-ojphi-04-17] Figure 1 

Study identification and selection process.


Dimensions of IS Integration

The difficulties and challenges arising from the integration of multiple technologies into a single IS were noted by six studies (Table 1); five of which were examinations of IIS. Heterogeneity was manifest as different methods of data collection,24 differences in the structure and storage of data between data sources,36 differences in the amount and quality of information collected from other systems based on technology,1820 and differences in the IIS’ ability to both send and receive information.15 While electronic data repositories in the form of electronic health records or billing systems tended to make data more accessible to the IIS and of higher quality,19,20 simply having an electronic data source was not a panacea. For example, Kolasa and colleagues18 reported that just because an electronic billing system was in place did not mean the IIS received all the data. Furthermore, each unique system from which an IIS obtains data requires its own interface, which requires time and money.15 Conversely, Schauer and colleagues22 noted the ability of each LHDs to exchange information in Wisconsin was a strength of the IIS.

[TableWrap ID: t1-ojphi-04-17] Table 1 

Characteristics and findings of research studies describing public health information system factors associated with quality and system integration, 2005–2011.


Citation System investigated Study design & subjects System integration issues Data quality issues
Immunization Information systems
Kolasa, Cherry, et al. (2005)18 Philadelphia Kids Immunizations Database/Tracking System Cross-sectional, review of IIS records & provider records Heterogeneity
  • Use of an electronic billing systems did not mean all data were transferred to IIS

Autonomy
  • Reporting rates varied by provider organization type & size
Completeness
  • 24% of provider recorded immunizations not reported in the IIS
Kolasa, Chilkatowsky, et al. (2006)19 Philadelphia Kids Immunizations Database/Tracking System Cross-sectional, review of provider records, & IIS records Heterogeneity
  • Differences in submission rates based on how records submitted: electronic medical records highest & electronic billing higher than paper records

Autonomy
  • Reporting rates varied by provider organization type: hospitals had highest rates
Completeness
  • Provider charts more complete than IIS on up to date status but varied by data entry type:
    • ○ IIS & direct entry: κ = 1.0
    • ○ IIS & electronic medical record: κ = 0.72
    • ○ IIS & electronic billing: κ = 0.66
    • ○ IIS & manual billing: κ = 0.65
    • ○ IIS & manual logs: κ = 0.42
  • Provider charts more complete than IIS on up to date status but varied by provider type:
    • ○ IIS & hospital-based: κ = 0.81
    • ○ IIS & pediatric practice: κ = 0.37
    • ○ IIS & family practice: κ = 0.63
    • ○ Overall : κ = 0.58
Dombkowski, Leung, et al. (2007)16 Michigan Care Improvement Registry Cross-sectional, survey of physicians Completeness
  • Majority of providers found IIS information complete

Ease of use
  • Few practices had difficulty in accessing IIS

Free-of-error
  • Majority of providers found IIS information accurate

Value-added
  • Concerns about patients limited to Medicaid only
Stecher, Adelman, et al. (2008)23 Arizona State Immunization Information System Cross-sectional, review of primary care records, parental recall, & IIS records Autonomy
  • Mandatory reporting to IIS for providers resulted in high percent of children included
  • Providers can choose to electronically report or mail reports
Completeness
  • IIS specificity for up to date: 74%
  • IIS sensitivity for up to date: 47%
  • II positive predictive value for up to date: 72%
  • 91% of children included in the IIS

Timeliness
  • Mailed reports may take up to 2 week to entered & electronic reports reviewed for 1-2 days before entered
Mahon, Shea, et al. (2008)20 Boston Immunization Information System Cross-sectional, review of pediatric clinic records Heterogeneity
  • Fewer discrepancies in data from practices with electronic medical records

Autonomy
  • Even with an electronic medical record, providers may not create electronic records for existing patients
Believability
  • Records of individuals older than the IIS may not be accurate

Completeness
  • Manufactures, lot numbers, doses, & dates missing in IIS

Free-of-error
  • Manufacture & lot numbers in provider records did not match IIS
White, Anderson, et al. (2009)24 Minnesota Immunization Information Connection Cross-sectional, review of hospital records, & interviews with hospital staff Heterogeneity
  • IIS data derived from birth certificates, direct data entry, & historical records

Autonomy
  • Hospitals were not providing vaccination data beyond what was required on birth certificates
Completeness
  • IIS missing birth doses administered in hospitals
Schauer, Maerz, et al. (2009)22 Wisconsin Immunization Registry Case study Heterogeneity
  • All LHDs could exchange data with the IIS

Distribution
  • Difficult to identify provider participation given complexity of provider networks and affiliations
Completeness
  • Data missing on children vaccinated by non-IIS users

Free-of-error
  • Records of children who have moved out of state not de-activated
Groom, Kennedy, et al. (2010)1 Various Interviews 7 state & urban area immunization program managers Distribution
  • Care obtained from multiple providers
Ease of use
  • Providers have quick access to IIS information
Dombkowski, Reeves, et al. (2011)17 Michigan Care Improvement Registry Cross-sectional, analysis of reminder / recall notifications sent by LHDs Timeliness
  • Outdated parental contact information
Papadouka, Metroka, et al. (2011)21 New York City Immunization Information System Cross-sectional, review of patient records by LHD staff Completeness
  • Only 37% of doses recorded in IIS contained lot numbers
Arzt, Forney, et al. (2011)15 New York City Citywide Immunization Registry Case study Heterogeneity
  • Requires multiple interfaces, “Each vendor requires a separate development effort and a significant investment of time and resources”
  • Primarily unidirectional data flow & few vendors prepared to implement bi-directional data flow
Vital Records Information Systems
Smith, Veazie, et al. (2005)35 Maryland vital records Cross-sectional, comparison of death certificates & multiple occupational injury fatality systems Free-of-error
  • Death certificates had a higher sensitivity than the other sources (89%), but still did not include all cases
Lydon-Rochelle, Holt, et al. (2005)32 Washington vital records Cross-sectional, comparison of birth certificates, hospital discharge data, & medical records Free-of-error
  • Birth certificate maternal labor & birth events:
    • ○ True positive fraction ranged from 34.4% to 81.2%
    • ○ False positive fraction ranged from 0.1% to 7.4%
    • ○ Positive predictive values ranged from 60.3% to 93.0%
    • ○ Negative predictive values ranged from 81.2% to 99.1%
Mann, Knight, et al. (2005)33 Utah vital records Cross-sectional, comparison of death certificates, hospital records, & emergency medical services records Free-of-error
  • Death certificates do not reflect all injury mortalities
Rodriguez, Mallonee, et al. (2006)34 Oklahoma vital records Cross-sectional, comparison of death certificates & injury surveillance system data on traumatic brain injury mortality Free-of-error
  • Sensitivity of death certificates = 78%
  • Positive predictive value of death certificates = 98%
  • Higher odds of missed cases for females & older individuals
  • Higher odds of missed cases by cause of death coding
Horon (2005)30 Maryland vital records Cross-sectional, comparison of death certificates, fetal death records, & medical examiner records Free-of-error
  • 38% of maternal deaths were unreported on death certificates
Lydon-Rochelle, Cárdenas, et al. (2005)31 Washington vital records Cross-sectional, comparison of fetal death records with medical records Free-of-error
  • Fetal death certificate maternal & perinatal conditions:
    • ○ True positive rate ranged from 0.0% to 100.0%
    • ○ False positive rate ranged from 0.0% to 10.5%
    • ○ Positive predictive values ranged from 0.0% to 100.0%
    • ○ Negative predictive values ranged from 33.3% to 100.0%
Caveney, Smith, et al. (2006)27 Texas vital records Cross-sectional, comparison of death certificates, medical records, & individual interviews Free-of-error
  • Death certificate race/ethnicity reporting consistent with self-reports
    • ○ Percent agreement: 97.1%
    • ○ Sensitivity: 95.4%
    • ○ Specificity: 99.0%
Fiscella & Meldrum (2008)28 California vital records Cross-sectional, comparison of death certificates & hospital discharge records Free-of-error
  • Agreement between hospital discharge records & death certificates variable by race:
    • ○ White: κ = 0.76
    • ○ Black: κ = 0.92
    • ○ Asian: κ = 0.88
    • ○ Native American: κ = 0.27
    • ○ Other: κ < 0.01
Brender, Suarez, et al. (2008)26 Texas vital records Cross-sectional, comparison of birth certificates & individual interviews Free-of-error
  • Birth certificates have high specificity with parental self-reported occupation
  • Sensitivity of birth certificates for maternal & parental varies by occupational classification group
Fitzgerald, Wartenberg, et al. (2009)29 50 states’ birth & fetal death records Document review of forms Relevancy
  • Most states did not collect information that would be useful for environmental exposure investigations:
    • ○ Duration of residence for mothers
    • ○ Paternal residence
    • ○ Parental occupation
Boulet, Shin, et al. (2011)25 Atlanta vital records Cross-sectional, comparison of birth certificates & Metropolitan Atlanta Congenital Defect Program records Free-of-error
  • Low sensitivity of birth certificates for birth defects
Multiple content systems
Chapman, Ford, et al. (2011)36 Virginia Vital Events and Screening Tracking System Case study Heterogeneity
  • Differences in data field types between systems

Autonomy
  • Organizations create different identifiers for the same individual

Regardless of what technology is used to share data with public health agencies, data sharing remained subject to variation introduced by individual users and organizations. Organizational and individual autonomy are illustrated at a very basic level by differences in the rate of data sharing by organizational type and size.18,19 The challenges posed by differences in organizational practice were illustrated in three studies. Specifically, organizations create different identifiers for the exact same individual,36 individuals within an organization may choose not to use an information system,20 or organizations may not collect all the data desired by public health organizations.24 As a potential solution to the problems posed by organizational and individual autonomy in data sharing, the Arizona State Immunization Information System reported a higher percentage of children were included in the IIS due to mandatory reporting for providers.23

Only two studies described how the number of data sources, or the problem of distribution, affected public health data sharing. A case study of the Wisconsin Immunization Registry noted the basic activity of identifying provider participation in the IIS was complicated by the number of networks and affiliations contributing data to the system.22 Also specific to data on immunizations, Groom and colleagues1 noted how the reality of patients seeking care from multiple providers can prevent public health from having complete information.

[TableWrap ID: t2-ojphi-04-17] Table 2 

Summary of Distribution of Issues by Assessment Framework (n=23 studies).


Dimensions of IS Integration Immunization Vital Records Multiple content
Heterogeneity 6    1
Autonomy 5    1
Distribution 2      
Data Quality Characteristics Immunization Vital Records   
Completeness 8      
Error Free 3 10   
Ease of use 2      
Timeliness 2      
Believability 1      
Value added 1      
Relevancy    1   

Data quality issues

The studies of vital record information systems (VRIS) were exclusively concerned with data quality and more specifically if the data were free-from-errors. Under the data quality assessment framework, free-from-error is roughly analogous to misclassification bias. So most studies were not concerned whether or not a vital event had occurred, but whether the data describing that event was error free. In terms of the cause of death, VRIS underreported injury mortality3335 and VRIS also did not accurately reflect pregnancy status30 on death certificates. Natality data in the VRIS were also the subject of reviewed studies, which concluded substantial variation on the recording of labor & delivery events,32 birth defect data were not free-from-errors,25 and parental occupations were not categorized correctly.26 The reviewed studies were not conclusive about the extent race/ethnicity data in VRIS are free-from-error. For example, in a comparison of California hospital discharge records and death certificates, Fiscella and Meldrum28 noted wide variation in the data agreement by race/ethnicity, but a comparison of death certificates with self-reported race/ethnicity found high agreement in Texas.27 Lastly, Fitzgerald and colleagues’29 warrants specific attention because of all the VRIS studies, it was the only one concerned with the relevancy of the data. Instead of assessing whether data were free-from-error or complete, the authors sought what types of additional data would be useful.

The reviewed articles examining IIS tended to be more diverse in their treatment of data quality issues than those focused on VRIS. However, if the data were complete and free-from-error was still a major focus. Studies reported their respective IIS were missing vaccination doses,1820,23,24 missing manufactures & lot numbers,20,21 or even whole individuals.22 Even when present, manufacture and lot numbers may be wrong20 or individuals may have moved out of the state.22 Other dimensions identified in the reviewed articles included: the believability of older records in IIS,20 the timeliness of the available contact information,17 and the ease of system use.1

A single article requires additional comment. Dombkowski and colleagues16 presented a more comprehensive evaluation of an IIS than the other studies by examining end user perceptions. While these were the perceptions of physicians users, the system was nonetheless a public health IIS. They reported positive perceptions of data completeness, that it was free-from-error, the system was easy to use and that it was beneficial to use in their work.


Discussion

The review of evidence on IS in public health, as revealed through immunization and vital records, confirms the general perception that significant challenges and barriers prevent public health from leveraging IS to its fullest extent.6 In general, this review revealed public health IS are struggling with issues of integrating multiple technologies, data sources and independent organizations and often not providing information of sufficient quality to public health practitioners. The IS qualities reported had bearing on public health practice, policy making and research. However, beyond simply documenting challenges, this review provides insights into the interrelated nature of system features and data quality issues and highlights the need to incorporate more end user perspectives.

While each dimension of system integration and data quality were assessed independently, the articles included in our review on IIS illustrate how one dimension can affect another. For example, autonomy may be highly correlated with system heterogeneity as was the case in Arzt and colleagues’15 report where independent organizations were making their own, and very different, technology vendor choices. Likewise, several studies noting examples where organizations limited data sharing, choose inefficient methods of exchange or did not participate fully in the system also reported issues around completeness.18,23,24 These instances of the importance organizational and individual decisions on an integrated IS, or at least a hoped for integrated IS, indicate both a priority and a role for public health. Although data quality is paramount for quality decision making, the priority in public health IS should be from the system perspective. Because public health deals with populations, if differing technologies are not rectified, if individual differences in organizational practices are not mitigated and if all components of the public health system not included, it will not be possible to compile data that is accurate, timely, complete and error-free. Without a systems focus, the dimensions of IS integration (heterogeneity, distribution and autonomy) are ignored and thus likely to present as longer term, but avoidable, problems.

To facilitate this approach, public health agencies need to take a strong coordinating role. We intentionally use the word coordination, because no public health entity will be able to eliminate organizational autonomy. Even in states with highly centralized governance structures, public health will not be able to dictate specific vendor choices or business processes. Current federal policy around electronic health records demonstrates that reality. What public health can do is coordinate around specific capabilities like exchange standards and required data elements, which is consistent with broader health IT policy, provide guidance to locally governed LHDs, or deploy uniform solutions within centrally governed states.

Previous research agendas in the area of public health IS noted the importance of attending to user needs.37,38 With a few exceptions (for example1,16) specific public health user perceptions or needs were not at the forefront of the reviewed literature. However, the examined literature revealed that public health data are valued as much for research purposes as for practice meaning the authors were primarily focused on researchers as their users of interest (for example20,27,28,30). A dichotomization between the practice and research value of public health information does not bode well for generating evidence relevant to understanding the real-time, daily needs of LHD staff, for participatory research with practitioners, nor for generating knowledge from practice. Researchers are more likely to be concerned with quality of data than on integration, compared to the concerns of practitioners. Both groups ought to be equally concerns about both aspects of IS.

This review documents the state of two public health IS examples over the past six years. However, public health IS will undoubtedly undergo a transformation over the course of this decade for two very different reasons. The first is essentially self-imposed: the launch of the public health accreditation process. Accreditation standards focus on data driven decision making and include requirements of data sharing with other public health organizations,39 which should foster interests and investment in public health IS. With the majority of state public health agencies intending to seek accreditation40 and not an inconsequential level of interest among LHDs,5 the potential for increased attentiveness to IS issues around the nation could increase substantially. The other development, which will have a multi-faceted impact, is the national effort to increase adoption of electronic health records under the Meaningful Use program.41 For LHDs providing primary care services, this program will provide unparalleled financial incentives to adopt a key, interoperable clinical information system thereby increasing the technological capacity for numerous public health organizations. For all of public health, the Meaningful Use criteria specific to public health reporting will dramatically increase existing distribution and heterogeneity challenges. Effectively all healthcare providers and organizations will soon be able, and required, to contribute data to public health. Public health will have unprecedented access to clinical data on essentially everyone who accesses the healthcare system, but it will come at the cost of a dramatic increase in number of organizations contributing to public health as well as the data originating from numerous technologies.


Conclusions

For state and local practitioners to effectively utilize data, public health IS will have to overcome the challenges posed by a large number of autonomous data providers utilizing a variety of technologies. Efforts to ensure quality information must remain attentive to the role of overarching system factors.


Acknowledgments

This work was supported by the Robert Wood Johnson’s Dissertation and Junior Faculty Awards in PHSSR in conjunction with the Public Health Services and Systems Research Coordinating center.

Appendix 
Search terms
  • Immunization information systems
  • Immunization registries
  • Vital records
  • Birth certificates
  • Death certificates
  • Birth records
  • Death records
Example search strategy for PubMed
  1. All field search of “immunization information systems”
  2. All field search of “immunization information system”
  3. All field search of “vital record system”
  4. All field search of “vital records system”
  5. All field search of “vital records systems”
  6. All field search of “vital record systems”
  7. #1 OR #2 OR #3 OR #4 OR #5 OR #6
  8. MESH term birth certificates
  9. MESH term birth records
  10. MESH term death certificates
  11. MESH term death records
  12. MESH term immunization
  13. #8 OR #9 OR #10 OR #11 OR #12
  14. MESH term information systems OR MESH term computer systems OR MESH term public health informatics
  15. #13 AND #14
  16. #15 OR #7
  17. Limits: English. Year 2005/1/1 – 2011/12/31.

Abstract Number retrieved = 518

Relevant journals, organization’s websites and conference abstracts
  • American Journal of Public Health
  • BMC Public Health
  • Journal of Public Health Management & Practice
  • Online Journal of Public Health Informatics
  • Public Health Reports
  • Centers for Disease Control & Prevention
  • National Association of County & City Health Officials
  • Association of State & Territorial Health Officials
  • National Organization of Urban Maternal & Child Health Leaders (CityMatCH)
  • Association of Maternal and Child Health Programs (AMCHP)
  • National Center for Health Statistics (NCHS)
  • Robert Wood Johnson Foundation
  • Annual meetings of the American Public Health Association from 2005 to 2011

References
1. Groom H, Kennedy A, Evans V, Fasano N. Qualitative Analysis of Immunization Programs With Most Improved Childhood Vaccination Coverage From 2001 to 2004Journal of Public Health Management and Practice 2010;16(1):E1–E8. [doi: 10.1097/PHH.0b013e3181b0b8bc]
2. Lumpkin JR, Richards MS. Transforming The Public Health Information InfrastructureHealth Affairs 2002;21(6):45–56. [doi: 10.1377/hlthaff.21.6.45]
3. Teutsch S. Considerations in Planning a Surveillance SystemLee L, Teutsch S, Thacker S, St Louis MPrinciples and Practice of Public Health Surveillance. New York: Oxford University Press; 2010
4. Vest JR, Menachemi N, Ford E. Governance’s role in local health departments' information system and technology usageJournal of Public Health Management & Practice. in press
5. National Association of County & City Health Officials2010 National Profile of Local Health Departments. Washington, DC: 2011
6. Turning Point National Excellence Collaborative for Information TechnologyFinal Report of the Collaborative. 2005
7. Association of State & Territorial Health OfficialsASTHO Profile of State Public Health, Volume 1. Arlington, VA: 2010
8. Linkins RW. Immunization registries: progress and challenges in reaching the 2010 national objectiveJ Public Health Manag Pract 2001;7(6):67–74. Epub 2001/11/17.
9. National Research CouncilVital Statistics: Summary of a Workshop. Committee on National Statistics DoBaSSaEWashington, DC: The National Academies Press; 2009
10. Pipino LL, Lee YW, Wang RY. Data quality assessmentCommun ACM 2002;45(4):211–8. [doi: 10.1145/505248.506010]
11. Lee YW, Strong DM, Kahn BK, Wang RY. AIMQ: a methodology for information quality assessmentInformation & Management 2002;40(2):133–46.
12. Hasselbring W. Information system integrationCommun ACM 2000;43(6):32–8. [doi: 10.1145/336460.336472]
13. National Association of County & City Health Officials2008 National Profile of Local Health Departments. Washington, DC: 2009
14. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementPLoS Med 2009;6(7):e1000097. Epub 2009/07/22 [doi: 10.1371/journal.pmed.1000097] PubMed [pmid: 19621072] PubMed Central PMCID: [pmcid: PMC2707599]
15. Arzt NH, Forney K, Chi A, Suralik M, Schaeffer P, Aponte A. Meaningful use and public health: an immunization information sysetm case studyJournal of Healthcare Information Management 2011;25(4):37–44.
16. Dombkowski KJ, Leung SW, Clark SJ. Provider attitudes regarding use of an immunization information system to identify children with asthma for influenza vaccinationJ Public Health Manag Pract 2007;13(6):567–71. Epub 2007/11/07 [doi: 10.1097/01.PHH.0000296131.77637.1f] PubMed [pmid: 17984709]
17. Dombkowski KJ, Reeves SL, Dong SM, Stevenson J, Clark SJ. Assessing the burden of undeliverable immunization reminder and recall notificationsPreventive Medicine 2011;53(6):424–6. [doi: 10.1016/j.ypmed.2011.09.014] PubMed [pmid: WOS:000298073500014]
18. Kolasa MS, Cherry JE, Chilkatowsky AP, Reyes DP, Lutz JP. Practice-based Electronic Billing Systems and Their Impact on Immunization RegistriesJournal of Public Health Management and Practice 2005;11(6):493–9.
19. Kolasa MS, Chilkatowsky AP, Clarke KR, Lutz JP. How complete are immunization registries? The Philadelphia storyAmbulatory pediatrics : the official journal of the Ambulatory Pediatric Association 2006;6(1):21–4. Epub 2006/01/31 [doi: 10.1016/j.ambp.2005.08.006] PubMed [pmid: 16443179]
20. Mahon BE, Shea KM, Dougherty NN, Loughlin AM. Implications for registry-based vaccine effectiveness studies from an evaluation of an immunization registry: a cross-sectional studyBMC Public Health 2008;8:160. Epub 2008/05/16 [doi: 10.1186/1471-2458-8-160] PubMed [pmid: 18479517] PubMed Central [pmcid: PMC2412875]
21. Papadouka V, Metroka A, Zucker JR. Using an Immunization Information System to Facilitate a Vaccine Recall in New York City, 2007Journal of Public Health Management and Practice 2011;17(6):565–8. [doi: 10.1097/PHH.0b013e3182214746] PubMed [pmid: WOS:000295400100016]
22. Schauer SL, Maerz TR, Hurie MB, Gabor GW, Flynn JM, Davis JP. The use of an immunization information system to establish baseline childhood immunization rates and measure contract objectivesJ Public Health Manag Pract 2009;15(5):E6–12. Epub 2009/08/26 [doi: 10.1097/PHH.0b013e3181a391ba] PubMed [pmid: 19704303]
23. Stecher DS, Adelman R, Brinkman T, Bulloch B. Accuracy of a state immunization registry in the pediatric emergency departmentPediatric Emergency Care 2008;24(2):71–4. [doi: 10.1097/PEC.0b013e318163db4d] PubMed [pmid: WOS:000253261700002]
24. White KE, Anderson J, Stanley M, Ehresmann K. Evaluating hepatitis B universal birth dose vaccination at Minnesota birthing hospitals by utilizing immunization information systems, birth certificates, and chart reviews, 2007–2008J Public Health Manag Pract 2009;15(6):464–70. Epub 2009/10/14 [doi: 10.1097/PHH.0b013e3181aab5e0] PubMed [pmid: 19823150]
25. Boulet SL, Shin M, Kirby RS, Goodman D, Correa A. Sensitivity of birth certificate reports of birth defects in Atlanta, 1995–2005: effects of maternal, infant, and hospital characteristicsPublic Health Reports (Washington, DC: 1974) 2011;126(2):186–94. PubMed [pmid: 21387948] Language Code: eng. Date Created: 20110310. Date Completed: 20110330. Update Code: 20111122. Publication Type: Journal Article. Journal ID: 9716844. Publication Model: Print. Cited Medium: Print. NLM ISO Abbr: Public Health Rep. PubMed Central ID: PMC3056031 [Available on 03/01/12] Linking ISSN: 00333549. Subset: AIM.
26. Brender J, Suarez L, Langlois P. Validity of parental work information on the birth certificateBMC Public Health 2008;8(1):95. PubMed PMID: [doi: 10.1186/1471-2458-8-95]
27. Caveney AF, Smith MA, Morgenstern LB, Lisabeth LD. Use of death certificates to study ethnic-specific mortalityPublic Health Rep 2006;121(3):275–81. Epub 2006/04/28. PubMed [pmid: 16640150] PubMed Central [pmcid: PMC1525286]
28. Fiscella K, Meldrum S. Race and ethnicity coding agreement between hospitals and between hospital and death dataMedical Science Monitor 2008;14(3):SR9–SR13. PubMed [pmid: WOS:000254101400018]
29. Fitzgerald E, Wartenberg D, Thompson WD, Houston A. Birth and fetal death records and environmental exposures: promising data elements for environmental public health tracking of reproductive outcomesPublic Health Rep 2009;124(6):825–30. Epub 2009/11/10. PubMed [pmid: 19894425] PubMed Central [pmcid: PMC2773946]
30. Horon IL. Underreporting of Maternal Deaths on Death Certificates and the Magnitude of the Problem of Maternal MortalityAmerican Journal of Public Health 2005;95(3):478–82. [doi: 10.2105/ajph.2004.040063]
31. Lydon-Rochelle MT, Cárdenas V, Nelson JL, Tomashek KM, Mueller BA, Easterling TR. Validity of Maternal and Perinatal Risk Factors Reported on Fetal Death CertificatesAmerican Journal of Public Health 2005;95(11):1948–51. [doi: 10.2105/ajph.2004.044305]
32. Lydon-Rochelle MT, Holt VL, Nelson JC, Cardenas V, Gardella C, Easterling TR, et al. Accuracy of reporting maternal in-hospital diagnoses and intrapartum procedures in Washington State linked birth recordsPaediatr Perinat Epidemiol 2005;19(6):460–71. Epub 2005/11/05 [doi: 10.1111/j.1365-3016.2005.00682.x] PubMed [pmid: 16269074]
33. Mann NC, Knight S, Olson LM, Cook LJ. Underestimating injury mortality using statewide databasesJ Trauma 2005;58(1):162–7. Epub 2005/01/28. PubMed [pmid: 15674167]
34. Rodriguez SR, Mallonee S, Archer P, Gofton J. Evaluation of death certificate-based surveillance for traumatic brain injury--Oklahoma 2002Public Health Rep 2006;121(3):282–9. Epub 2006/04/28. PubMed [pmid: 16640151] PubMed Central [pmcid: PMC1525278]
35. Smith GS, Veazie MA, Benjamin KL. The use of sentinel injury deaths to evaluate the quality of multiple source reporting for occupational injuriesAnn Epidemiol 2005;15(3):219–27. Epub 2005/02/23 [doi: 10.1016/j.annepidem.200407.094] PubMed [pmid: 15723768]
36. Chapman DA, Ford N, Tlusty S, Bodurtha JN. Evolution of an integrated public health surveillance systemJ Registry Manag 2011;38(1):15–23. Epub 2011/11/22. PubMed [pmid: 22097701]
37. Yasnoff WA, Overhage JM, Humphreys BL, LaVenture M, Goodman KW, Gatewood L, et al. A national agenda for public health informaticsJ Public Health Manag Pract 2001;7(6):1–21. Epub 2001/11/21. PubMed [pmid: 11713752]
38. Kelly JS, Zimmerman LA, Reed K, Enger KS. Immunization information systems national research and evaluation agendaJ Public Health Manag Pract 2007;13(1):35–8. Epub 2006/12/07. PubMed [pmid: 17149098]
39. Public Health Accreditation BoardStandards & Measures Version 1.0 Alexandria, VA2011 [10 April 2012] Available from: http://www.phaboard.org/wp-content/uploads/PHAB-Standards-and-Measures-Version-1.0.pdf.
40. Association of State & Territorial Health OfficialsASTHO Profile of State Public Health, Volume 2. Arlington, VA: 2011
41. Department of Health & Human Services42 CFR Parts 412, 413, 422 et al. Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Final RuleFederal Register 2010;75(144):44314–588.

Article Categories:
  • Articles

Keywords: public health informatics, registries, birth records, information systems, vaccination, organization and administration.




Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org