Aggregate poll Web site use across the 2016 United States presidential election
First Monday

Aggregate poll Web site use across the 2016 United States presidential election by Toby Hopp and Kim Sheehan

This study examined citizens’ use of poll aggregation Web sites over the last nine weeks of the 2016 United States presidential campaign. The results suggested that usage frequency increases as election day approaches. Moreover, those with high levels of political interest and those that actively use media to obtain political information are most likely to access poll aggregation Web sites.


Theoretical background
Discussion and conclusions




Web sites that aggregate individual political poll surveys have become an increasingly popular facet of the media coverage surrounding American election campaigns. Examples of these Web sites include FiveThirtyEight, Real Clear Politics, and Huffington Post’s HuffPollster. In contrast to poll providers such as Ipsos, Gallup, and Harris Interactive, poll aggregation Web sites combine multiple poll surveys in order to better describe the state of an election. Despite both the popularity and utility of poll aggregation Web sites, relatively little is known about either reader usage patterns or the civic factors that drive engagement. Accordingly, this study employed a longitudinal design to pursue two research goals. First, this study set out to better understand how people use/consume aggregate poll Web sites across the course of a presidential election. Second, the present study modified Dahlgren’s (2000) civic culture dimensions to investigate the relationship between one’s civic profile and use of poll aggregation sites across the 2016 presidential election cycle.



Theoretical background

Poll aggregation Web sites

Error is inherent in individual poll surveys. As pointed out by Pasek (2015), when conducting public opinion surveys, polling firms must make critical decisions “about how and when people should be surveyed, which respondents should be classified as likely voters, and how sampled individuals should be weighted to represent the public” [1]. These decisions, combined with factors such as measurement and random error, can result in a great deal of incongruence across individual survey estimates.

Given issues pertaining to the accuracy of individual poll survey estimates, polling aggregators have experienced a surge of popularity over the past decade (Hillygus, 2011). Specifically, poll aggregators seek to combine multiple poll surveys in order to arrive at “the best possible objective assessment of the likely outcome of upcoming elections” [2]. By examining multiple inputs, poll aggregators are provided a better means of addressing error and, therefore, can presumably arrive at better estimates of population-level beliefs, attitudes, and voting intentions [3]. Indeed, as illustrated by Wang (2015), “polling aggregators have been outperforming pundits since at least 2004, when a number of Web sites began to collect and report polls on a state-by-state basis in Presidential, Senate, and House races” [4].

Notably, poll aggregation Web sites provide an opportunity for serial engagement, especially during high-profile election campaigns. Poll aggregation Web sites are consistently updated with new information, often multiple times per day. For interested readers, this provides motivation for repeated visits throughout the lifespan of the election, effectively allowing for the emergence of a coherent and identifiable group of dedicated users. Likewise, news organizations can use new and/or emergent analysis from these sites to both contextualize and frame their ongoing coverage of the election. As illustrated by Toff (2017), the estimates provided by poll aggregation Web sites such as FiveThirtyEight conform with the goals of precision journalism, and have, therefore, occupied an increasingly influential place in election-related reporting.

Despite their increased use by both citizens and journalists, little is known about citizen-level engagement with aggregate polling Web sites. As aggregate polling sites increasingly become part of election-related attentional repertoires, it is useful to understand usage patterns across the course of the election period.

RQ1: How does use of poll aggregation Web sites vary across the course of the 2016 presidential election campaign?

Civic culture as a means of understanding individual patterns of aggregate poll Web site use

The basic contention behind Dahlgren’s (e.g., 2000) civic culture perspective is that “features of the socio-cultural world ... constitute everyday pre-conditions for all democratic participation” [5]. Dahlgren distinguished between the formal, systems-based aspects of governance and the cultural, normative aspects of civic participation, and, in so doing argued that these two aspects of civic life, while distinct, naturally “evolve in relation to one another” [6]. Considered specifically within the context of emergent democratic technologies, the civic culture perspective provides a means of interrogating the relationship between online and off-line forms of civic participation. Speaking broadly on the topic of the Internet, Dahlgren [7] previously remarked:

The political trends in modern democracy articulate in complex ways with the evolution of the media. The dimensions of civic culture offer ways to organize analyses of how the media, via their modes of representation as well as the newer forms of interactivity that they offer, are possibly contributing to the decline of traditional political life and the emergence of newer forms of involvement.

Civic culture has four elements: (1) knowledge and competencies; (2) loyalty to democratic values; (3) citizen practices, traditions and routines; and (4) citizen identities. The knowledge and competency dimension refers to both citizen access to important political information and the individual-level resources necessary to access and make sense of such information (Dahlgren, 2005). Loyalty to democratic values refers to a commitment to democratic principles such as deliberation and compromise (Dahlgren, 2002). The dimension pertaining to citizen practices and routines describes adherence to critical civic practices such voting and the pursuit of good-faith political discussion. Finally, the identity component describes both one’s formal designation as a citizen (Dahlgren, 2000) and the degree to which a citizen identifies with a given political party, movement, or ideology (Dahlgren, 2011).

The civic culture model is a constructionist device broadly designed to describe the expansive and dynamic forces that influence the relationship between the individual and the broad systems of democracy within which the individual is ensconced. This expansiveness does not lend itself to more granular forms of empirical investigation, which require both parsimony and operational clarity. That said, the dimensions offered by Dahlgren offer a helpful means of exploring how the features of one’s self as citizen predict his or her adoption and use of new forms of political information. Thus, in the present study, we employ the notion of civic culture as a starting point for empirical interrogation of the relationship between the various features of the individual as a citizen and his or her use of poll aggregation Web sites. Specifically, we use the four dimensions of Dahlgren’s model to construct individual-level civic profiles, or snapshots of the various features that describe people as citizens. Three specific dimensions are presented: identification, attention, and behavior. The identification dimension encompasses the notion of citizen identity, and pertains to both one’s partisan affiliation and the degree to which he or she adheres to the ideological underpinnings associated with each party. The second dimension, attention, is derived from Dahlgren’s knowledge and competencies dimensions, and specifically refers to citizen interest in and attention to political issues and events. Finally, the third dimension, behavior, touches upon both the loyalty and practice-based dimensions in Dahlgren’s model. This dimension specifically assesses the degree which a citizen enacts democratic values as part of their day-to-day life.

In light of the foregoing, the second research question pertains to the relationship between the features of one’s self as a citizen and the degree to which poll aggregation sites were used over the course of the 2016 presidential election. This line of inquiry will allow for a better understanding of the citizenship-based correlates of poll aggregation Web site usage, and, in so doing, help to better contextualize how these sites contribute to the society-wide body of election related knowledge.

RQ2: What aspects of one’s citizen profile predict use of poll aggregation Web sites over the course of the 2016 presidential election?




The present study was longitudinal in nature. Data were collected in three waves, spaced three weeks apart. Initial contact (T1) was made in the period between 23 September and 25 September 2016. Re-contact (T2) with those who participated at T1 was made three weeks later (16–18 October). The third wave (T3) of data collection began the day after the election (9 November) and closed on 11 November. As in the case of T2, T3 eligibility was determined on the basis of participation at T1. In all, the measurement points occurred at the beginning of campaign’s high saliency point, between the second and third presidential debates, and immediately after the election’s conclusion.

Data collection was accomplished through the use of TurkPrime, a managerial interface that draws participants from the Amazon Mechanical Turk (MTurk) population. Once an initial sample has been constructed, TurkPrime allows researchers to set future sample qualification criteria to include only those previously surveyed. Additionally, TurkPrime offers the ability to e-mail prior participants to inform them of the release of a new survey. TurkPrime also provides researchers a number of controls that help ensure sample quality. In the current study, T1 verification controls we set to ensure that participant IP addresses were from the United States. Additionally, to qualify for the survey, participants (at T1) were required to indicate that they were U.S. citizens, pass a basic test on American politics (respondents were asked to indicate the number of years that the president of the United States is elected for), indicate interest in voting in the 2016 presidential election, and be at least 18 years old. Taken as a whole, these efforts resulted in a focus on likely voters, whom we presumed to be the general population interested in political media generally and poll aggregation sites specifically. Valid responses acquired at each measurement point (T1–T3) were 699, 554, and 519, respectively. This study focused on the 464 participants who provided data on all three occasions.


Aggregate poll Web site use. To ensure that respondents possessed a common understanding of the media behavior under study, the survey contained brief description of poll aggregation sites (i.e., Web sites that collect individual poll estimates as a means of providing a “snapshot” of the current state of an electoral race). The survey tool also provided users with examples of prominent aggregate polling sites (Real Clear Politics, FiveThirtyEight, Huff Post Pollster, and Talking Points Memo’s Poll Tracker). This information was presented before respondents addressed any questions pertaining to media use. Aggregate polling Web site use itself was assessed at all three measurement points. In each case, two items were used to evaluate Web site usage: Thinking back over the last three weeks, about how often would you say that you have visited presidential polling Web sites such as Real Clear Politics or FiveThirtyEight)? and Thinking back over the last three weeks, about how often would you say that you have engaged with presidential polling Web sites such as Real Clear Politics or FiveThirtyEight? Response categories were placed on semantic differential scales where 1=very infrequently and 7=very frequently [8].

Citizen profile variables. The identification dimension was measured by assessing each respondent’s party ID, overall conservatism, political extremity, and who they voted for in the 2016 presidential election. Party ID was assessed at T1, and asked respondents to select their political affiliation (1=democrat, 2=republican, 3=independent, 4=other). For the purposes of data analysis, these measures were subsequently collapsed into dummy variables where democratic affiliation was set as the contrast category (0). Respondent conservatism was measured at T1 using two items (Generally speaking, what is your political ideology? and Generally speaking, I tend to support political candidates who are ...), both on 11 point semantic differential scales were 1=strongly liberal and 11=strongly conservative. Next, a measure of political extremity was created by recoding each of the above-described T1 conservatism measures. For each measure, those that indicated stronger ideological preference in either direction were assigned higher numbers, while those that were neutral/moderate were assigned lower numbers. The scale of this measure ranged from 1 (ideologically moderate) to 6 (ideologically extreme). Finally, at T3, respondents were asked to indicate who they voted for. For data analysis, this variable was collapsed into three categories: 0 = voted for Hillary Clinton, 1 = Donald Trump, 2 = voted for different candidate/didn’t vote/prefer not to indicate. Subsequently, this variable was transformed into two dummy variables with Clinton set as the contrast category (0).

To assess the interest factor, general political interest was measured at T1 using four items, all placed on seven-point Likert-type scales where 1 = strongly disagree and 7 = strongly agree (example statements: I’m interested in politics and I like to learn as much as I can about politics). Also at T1, participants’ political media use was assessed by asking the frequency with which they consumed hardcopy/online news, watched political programming on television, sought out political content on social media, and read political blogs (1=infrequently, 7=very frequently). These items were subsequently collapsed into a single indexed measure.

To assess the behavioral dimension, we measured online and off-line civic engagement at T1. Off-line civic engagement was measured using modified versions of the items presented in Gil de Zúñiga and Valenzuela (2011) and in McFarland and Thomas (2006). All scale items were placed on seven-point scales where 1 = never and 7 = frequently. Sample items include: Over the past 12 months, about how often have you worked or volunteered for political organizations?; and Over the past 12 months, about how often have you donated money to an organization whose mission you believe in? For its part, online civic engagement was measured using 5 items developed from Dozier, et al. (2016). As in the measure of off-line civic engagement, all items were on seven-point Likert-type scale where 1=never and 7=frequently. Sample items included: Over the past 12 months, about how often have you signed an online petition in support of a cause you believe in? and Over the past 12 months, how often have you followed or liked a political figure on social media? Additionally, at T3, we asked whether or not respondents voted in the 2016 presidential election (0=no, 1=yes).

Control variables. Age was measured at T1 and represented participant’s age as of their most recent birthday. Biological sex was measured at T1. Response categories were 0=female and 1 = male. For race, participants were asked (at T1) to indicate the racial/ethnic group they most strongly identify with. Finally, at T1, respondents were asked to provide an estimate of their annual household income. Response categories ranged from 1=US$20,000 or less to 11=US$200,000 or greater.

Descriptive statistics for the continuous measures described above are provided in Table 1. Frequencies for categorical variables are provided in the sample description below.


Descriptive statistics for continuous measures


Missing data analysis

In all, 44 cases (9.5 percent of the sample) were missing at least one value. Patterns of missingness in the data were examined using Little’s MCAR test. The results of this test (χ2=299.195, df=296, p>.05) suggested that missing values were randomly distributed (in other words, missing completely at random). Thus, the analyses reported in this manuscript were calculated using only complete cases (n=420) [9].

Sample description

The analytic sample was 54.8 percent female. On average, respondents were 37.99 years old (S = 11.80). In terms of race, 7.1 percent identified as Asian/Asian-American, 7.4 percent identified as Black/African-American, 4.3 percent identified as Hispanic/Latino, 78.1 percent identified as White/Caucasian, 0.5 percent identified as Native American/Native Hawaiian, 2.4 percent identified with more than one racial/ethnic group, and 0.2 percent identified with a racial/ethnic group not provided in the survey’s list of options. As it pertained to estimated annual income, the median reported value was between US$40,001 and US$60,000 annually (M=3.42, SD=1.99). In all, 93.3 percent of the sample reported annual household income levels of US$120,000 or less. At T1, 44.3 percent of the sample reported identifying as a Democrat, 31.2 percent identified as an Independent, 20.5 percent identified as a Republican, and 4.1 percent identified with some other party or political group. At T3, 90.2 percent indicated that they voted in the 2016 presidential election. Finally, at T3, 52.1 percent voted for Hillary Clinton, 26.4 percent voted for Donald Trump, and 21.4 percent either voted for a different candidate, did not vote, or chose to not disclose who they voted for.

Analytic plan

To address RQ1, a series of paired samples t-tests were used to examine point-to-point changes in the use of poll aggregation Web sites across the study period. Additionally, a growth model was used to assess (1) average intra-individual change in website usage over the study period; and, (2) the degree to which such change was variable across individuals. For the purposes of specification, T1 poll Web site usage was set as the intercept. The intercept and slope terms were allowed to freely co-vary. Next, to assess RQ2, we employed three OLS models that regressed usage levels at each measurement point on the control, identification, interest, and behavioral variables. Finally, using the growth model estimated to asses RQ1, the slope term (i.e., average rate of change) was regressed upon the control, identification, interest, and behavioral variables as a means of better understanding inter-individual change.




Research Question 1

Paired samples t-tests indicated that aggregate poll usage increased from T1 to T2, ΔM=0.47, t(419)=7.74, p<.001, d=0.39; from T2 to T3, ΔM=0.61, t(419)=8.69, p<.001, d= 0.43; and from T1 to T3, ΔM=1.07, t(419)=12.817, p<.001, d=0.66.

As it pertained to the latent growth model, the model-data fit was acceptable, χ2=1.84, df=1, p>.05; CFI>.99; RMSEA=.05 [90%CI=.00, .15]; SRMR=.01. The slope term was significantly different from 0, µs=0.53, se=0.04, p<.001, and a significant variance estimate was associated with the slope term, Ds=0.55, se=0.09, p<.001.

Research Question 2

Regarding factors associated with use (see Table 2), an OLS regression model indicated that males (b=0.53, p<.001), those with higher incomes (b=0.07, p<.05), people with higher levels of habitual political media use (b=0.19, p<.01), and those who use the Internet for civic purposes (b=0.19, p<.05) were significantly more likely to use poll aggregation Web sites at T1. At T2, an OLS regression analysis suggested that use of poll aggregation Web sites was highest among men (b=0.51, p<.01), those with higher incomes (b=0.10, p<.05), those with heightened political interest (b=0.17, p<.05), those with higher levels of habitual political media use (b=0.20, p<.01), those who are civically engaged off-line (b=0.17, p<.05), and those who use the Internet for civic purposes (b=0.19, p<.05). At T3, usage of poll aggregation websites was again higher among men (b=0.50, p<.05) and those with higher incomes (b=0.13, p<.01). The data further suggested that poll aggregation Web site usage was negatively associated with age (b=-0.02, p<.05) and positively associated with political interest (b=0.27, p<.01) and habitual political media use (b=0.19, p<.05). In contrast to T1 and T2, a statistical relationship between online civic engagement and poll aggregate use was not observed at T3 (b=0.15, p<.12).

The latent growth model was a good fit for the data, χ2=21.92, df=17, p>.05; CFI=.99; RMSEA=.02 [90%CI=.00, .05]; SRMR =.01. In examining variable coefficients associated with the slope term, the data showed older respondents were increasingly less likely to use poll aggregation Web sites (b=-0.01, p<.01), those with high incomes were increasingly likely to use poll aggregation Web sites (b=0.03, p<.05), those with high levels of political interest were increasingly likely to use poll aggregation Web sites (b=0.09, p<.05), and independents (in contrast to Democrats) were less likely to indicate increased use of poll aggregation Web sites over the election period (b=-0.29, p<.05).


Unstandardized coefficients for relationships between citizen profile variables and poll aggregation website usage factors
Note: Larger version available here.




Discussion and conclusions

The present study had two primary objectives. First, it set out to better understand the use of poll aggregation Web sites over the course of a presidential election campaign. Second, as a means of exploring the civic uses and potential of poll aggregation Web sites, we examined how elements of one’s civic profile influence the use of poll aggregation Web sites.

As it pertains to sample-wide usage patterns (RQ1), the present data indicate that poll aggregate Web site usage frequency increases as election day approaches. Specifically, the results of point-to-point t-tests and a latent growth curve model together indicate that poll aggregation Web site usage linearly increases across the course of the election campaign and that usage levels are highest in the latter stages of the campaign. Together, these findings suggest that as the election contest becomes increasingly salient, citizens may increasingly turn to poll aggregation Web sites to understand the state of the race. This assertion is consistent with prior research (e.g., Tewksbury, 2007), which has shown that political media usage is responsive to campaign milestones.

Examination of the OLS regressions (Table 2) used to predict Web site usage at each measurement point suggests that involvement of with poll aggregation tools may be primarily linked to the interest component of one’s civic profile. Specifically, we found that those with high levels of political interest and those that actively employ the media as a means of obtaining political information are most likely to access poll aggregation Web sites. Moreover, our data suggest that use of poll aggregation Web sites may also be associated with the broader use of the Internet for civic purposes.

Taken as a whole, the findings reported in Table 2 can be used to paint a rough picture of who uses poll aggregation Web sites. Our data suggest that users tend be male, to have higher incomes, and to possess high levels of political interest. Political interest appeared to play an especially important role in the use of poll aggregation Web sites. Looking specifically at inter-individual growth patterns, our data indicated that those with high levels of political interest at T1 were increasingly likely to access and use poll aggregation Web sites over the course of the presidential election.

The current work has a number of limitations. First, while longitudinal research can point to the existence of causal relations between constructs and, therein, give researchers a means of assessing the stability of between-variable relationships, it cannot be taken as wholly indicative of causality (i.e., we cannot claim that political interest causes increased Web sites usage). Second, the present study employed convenience sampling and the imposed screening criteria resulted in a sample that was generally interested in politics (i.e., self-reported likely voters). Finally, the posture of the current study was purposefully exploratory in nature. This study is, to the best of our knowledge, the first to investigate either the temporal or individual-level features that are associated with aggregate poll Web sites use. To that end, we modified a very general perspective on civic life (Dahlgren, 2000) to crudely identify areas of overlap between citizen practices and use of an emergent technology that, itself, possesses potential democratic implications. Future research should build upon these findings by more precisely examining the civic causes and democratic effects associated with use of poll aggregation Web sites. End of article


About the authors

Toby Hopp is Assistant Professor in the Department of Advertising, Public Relations, and Media Design in the College of Media, Communication, and Information at the University of Colorado Boulder. His research interests are broadly related to the uses and effects of digital and interactive media, the social and motivational factors that underlie uncivil online communication, and organizational transparency.
Direct comments to: tobias [dot] hopp [at] colorado [dot] edu

Kim Sheehan is Professor in the School of Journalism and Communication at the University of Oregon. She teaches courses that bridge the gap between communication theory and media practice. Her research involves culture and new technology, and she has published extensively about social media, online privacy, green advertising, advertising ethics, and direct-to-consumer prescription drug advertising.
E-mail: ksheehan [at] uoregon [dot] edu



1. Pasek, 2015, p. 595.

2. Silver, 2008, paragraph 3.

3. See Pasek, 2015, for discussion of the various approaches used by poll aggregators.

4. Wang, 2015, p. 899.

5. Dahlgren, 2000, p. 336.

6. Dahlgren, 2000, p. 335.

7. Dahlgren, 2000, p. 339.

8. The T1 questionnaire asked participants to estimate how often they accessed poll aggregation Web sites over the past three weeks. A dropdown menu that gave response categories ranging from 0 (never) to 22 (more than once per day). A majority of respondents (65.71 percent) responded that they had not accessed aggregate polling Web sites, resulting in an overdispersed count usage measure (M=2.66, SD=3.56). A non-parametric correlation analysis indicated that the two-item usage measure was strongly correlated with estimated number of Web site visits, rτ=0.82, p<.001 [Pearson’s r=.78, p<.001].

9. All descriptive statistics reported in this manuscript were derived using data from the complete case dataset (n = 420).



Peter Dahlgren, 2011. “Young citizens and political participation: Online media and civic cultures,” Taiwan Journal of Democracy, volume 7, number 2, pp. 11–25, and at, accessed 10 December 2017.

Peter Dahlgren, 2005. “The Internet, public spheres, and political communication: Dispersion and deliberation,” Political Communication, volume 22, number 2, pp. 147–162.
doi:, accessed 10 January 2018.

Peter Dahlgren, 2002. “In search of the talkative public: Media, deliberative democracy, and civic culture,” Javnost — The Public, volume 9, number 3, pp. 5–25.
doi:, accessed 14 January 2019.

Peter Dahlgren, 2000. “The Internet and the democratization of civic culture,” Political Communication, volume 17, number 4, pp. 335–340.
doi:, accessed 14 January 2019.

David M. Dozier, Hongmei Shen, Kaye D. Sweetser, and Valerie Barker, 2016. “Demographics and Internet behaviors as predictors of active publics,” Public Relations Review, volume 42, number 1, pp. 82–90.
doi:, accessed 14 January 2019.

Homero Gil de Zúñiga and Sebastián Valenzuela, 2011. “The mediating path to a stronger citizenship: Online and offline networks, weak ties, and civic engagement,” Communication Research, volume 38, number 3, pp. 397–421.
doi:, accessed 14 January 2019.

D. Sunshine Hillygus, 2011. “The evolution of election polling in the United States,” Public Opinion Quarterly, volume 75, number 5, pp. 962–981.
doi:, accessed 14 January 2019.

Daniel A. McFarland and Reuben J. Thomas, 2006. “Bowling young: How youth voluntary associations influence adult political participation,” American Sociological Review, volume 71, number 3, pp. 401–425.
doi:, accessed 14 January 2019.

Josh Pasek, 2015. “Predicting elections: Considering tools to pool the polls,” Public Opinion Quarterly, volume 79, number 2, pp. 594–619.
doi:, accessed 14 January 2019.

Nate Silver, 2008. “Frequently asked questions,” at, accessed 10 December 2017.

David Tewksbury, 2007. “Exposure to the newer media in a presidential primary campaign.” Political Communication, volume 23, number 3, pp. 313–332.
doi:, accessed 14 January 2019.

Benjamin Toff, 2017. “The ‘Nate Silver’ effect on political journalism: Gatecrashers, gatekeepers, and changing newsroom practices around coverage of public opinion polls,” Journalism (15 September).
doi:, accessed 14 January 2019.

Samuel S.–H. Wang, 2015. “Origins of presidential poll aggregation: A perspective from 2004–2012,” International Journal of Forecasting, volume 31, number 3, pp. 898–909.
doi:, accessed 14 January 2019.


Editorial history

Received 12 February 2018; accepted 14 January 2019.

Creative Commons License
“Aggregate poll Web site use across the 2016 presidential election” by Toby Hopp and Kim Sheehan is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Aggregate poll Web site use across the 2016 United States presidential election
by Toby Hopp and Kim Sheehan.
First Monday, Volume 24, Number 2 - 4 February 2019

A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2019. ISSN 1396-0466.