Performance on tests of economic literacy: A comparison of face–to–face with online instruction
First Monday

Performance on tests of economic literacy by Frank P. Albritton, Jr.

Does taking a college level course in the principles of economics improve economic literacy? While there is debate on whether courses in the principles of economics as taught at colleges and universities improve general economic literacy, more and more of these courses are being taught online. There has been considerable research in other subjects as to whether there is a difference in student performance dependent upon whether the courses was taken in a traditional face–to–face format or online. However, in the critically important subjects dealt with in the principles of economics, there has been discussion but little research conducted about whether online delivery is equivalent in terms of student achievement to the traditional face–to–face delivery. The purpose of this paper is to determine if there is a difference in the performance of students, as measured by economic literacy, between students taking a course in the principles of economics using a face–to–face versus online format. The data was collected using an online questionnaire over two semesters at a community college and analyzed using SPSS, version 12.


Review of literature
Discussion and conclusion




Does taking a course in economics, whether at the high school or college level, have any impact on increasing economic literacy? Gleason and van Scyoc (1995) discovered through administering 1,300 questionnaires of the Test of Economic Literacy that test scores of adults who had an economics course in high school scored statistically the same as adults who had had no economics education in high school. In a study by Walstad and Allgood (1999), it was found in a random sample of 300 college seniors that students who had taken an economics course in college scored only 62 percent on a 15–question economics knowledge–based test. This would barely be a passing grade using a normal scale. Walstad and Allgood also found that only three percent of students who had taken an economics course rated their knowledge of economics as excellent while 79 percent rated their knowledge as fair to poor. Given the overall lack of economic literacy among the general population, even among those who have taken an economics course, a key question is whether economic literacy among college students will be further negatively affected if students take courses using modalities other than face–to–face.



This paper will test the hypothesis that there is no statistically significant difference in economic literacy among students at a community college who complete a face–to–face course in economics as compared to students who complete the course using an online format. A comparison will be made between students who took the principles of economics with the same professor at a community college, with similar assignments and learning tools, who differed only in the modality in which the courses was taken, with one group taking the course online and the other taking the course in the traditional face–to–face setting.



Review of literature

An operational definition of economic literacy must be established in order to see the importance of and success or failure of attaining economic literacy through courses in the principles of economics taken in the traditional college setting. The types of tests used and the results of these tests given at several levels during the span of a person’s education and employment including high school, college, adult and place of employment, will be discussed. The advent of online education and its impact on economic literacy will also be explored.

Economic Literacy
Many economists feel that current courses in the principles of economics do not improve on economic literacy (Gleason and van Scyoc, 1995; Hansen, et al., 2002: Rader, 1996; Zimmerman, 2004). These courses are often taken during the freshman and sophomore years in college by students who are either completing the prerequisites for studying business or economics or completing the general education requirements of an associate or bachelor’s degree. This can be a critical error as according to Rader,

“Ignorance about the workings of an economic system can lead to attitudes that may be destructive of the very system that produced the economic well–being of members of that society.” 1]

The principles of economics courses as covered in most colleges and universities, attempts to convey to students the principles of the optimization principle which deals with making decisions by comparing costs and benefits. After the introduction of such concepts as opportunity cost and supply and demand, the principles courses attempt to prepare students for the more complex part of Rader’s definition, namely including an understanding of the costs versus benefits of government intervention in the market under antitrust laws in a case of market monopolization or an understanding of why the Federal Reserve should raise interest rates to combat inflation. The very nature of an understanding or misunderstanding of the importance of profit to an economic system can be critical. Profit in an industry is a necessary signal to the economic system that it should allocate resources (firm entry) to this sector. The very act of firm entry will lead to a mitigation of these higher levels of profit. Stern (2002), in his role as President of the Federal Reserve Bank of Minneapolis, also stressed the importance of economic literacy:

“Conveying economic ideas clearly is a very difficult thing to do, and yet it is essential that we succeed because too much of what passes for debate on policies is nearly incoherent. Certainly, citizens better steeped in the principles of economics would be able both to understand and to contribute to discussion about policy at a higher level, and consequently we should expect better policies over time as a result.” [2]

Where are the ideas which lead to economic literacy being conveyed? Watts (1998) cites information that only approximately two percent of all college students major in economics; at the secondary level, perhaps 30 to 50 percent take at least one course in economics. Thus, the majority of the population will not have had a course in economics at either the secondary or postsecondary level.

Studies have corroborated the lack of knowledge of economic literacy basics. Two major instruments which have been developed to assess economic literacy are the Test of Economic Literacy (TEL), developed by the Joint Council on Economic Education and The Standards in Economics Survey developed by the National Council on Economic Education (NCEE). Using the TEL, Gleason and van Scyoc (1995) tested both adults and high school students as to their economic literacy. The adults in the survey answered approximately 53 percent of the questions correctly while the high school students answered approximately 43 percent of the questions correctly. Using The Standards in Economics Survey, Louis Harris & Associates, Inc., found similar results when it conducted a survey of over 1,000 adults and over 1,000 high school students in grades 9 through 12. They found that adults scored an average of 57 percent on their understanding of basic economic concepts and high school students scored an average of 48 percent (National Council on Economic Education. 2005). Wood and Doyle (2002) reported on a survey of over a 1,000 corporate employees using the same questionnaire and found that the average score was a 67 percent.

Online Economic Education
Distance education enrollments are climbing at the rate of 30 percent per year compared to total college enrollment increases of approximately one to two percent per year (Bertagnoli, 2001). In 2002, 1.6 million students took an online college course in the United States while almost 600,000 of these students took all of their courses online (Allen and Seaman, 2003). This comes in light of studies such as a metastudy conducted by Allen, et al. (2002) which indicated that there was a slight preference for taking courses in the traditional face–to–face format.

While faculty have mixed views concerning the effectiveness of online education (DeLoughry, 1995; Labay and Comm, 2003), a study by Russell (2003) demonstrated that most research indicated no difference in outcomes between students taking distance learning courses and traditional courses thus Russell’s (1999) often cited phrase of the no significant difference phenomenon. A meta–analysis by Allen (2003) also found that there was no difference in the area of student satisfaction between traditional courses and distance learning courses. Wang and Newlin (2002) conducted a study which illustrated that one of the most significant reasons for success of students in the online environment comes from the self–selection by students who take online classes, namely students who successfully take and complete online classes are interested and have a positive attitude about learning and teaching online.

Although economic literacy is important to the success of the economic system, the level of economic literacy attained in the general population is not impressive. With the implementation of new forms of teaching and learning, it is important to ascertain whether economic literacy will be affected by the fastest growing of these methods, online learning.

With the objective of testing whether improvement in economic literacy was achieved in both face–to–face and online modalities, a pre–test/post–test methodology was employed using the National Council of Economic Education’s twenty questions (see Appendix) to test whether there was an improvement in economic literacy after the completion of a course in the principles of economics. The questions test several critical areas including allocation of scarce resources, markets, factors of production and economic institutions (Hansen, et al., 2002). A pre–test/post–test methodology has been used in several studies to measure economic literacy and is reliable way to measure an increase in economic literacy (Zimmerman, 2004).




Students who registered for principles of economics classes, both face–to–face and online sections, for a specific instructor at a Central Florida community college over two semesters during the 2003/2004 academic year will be compared using a pre– and post–test methodology. While 57 percent of students at this community college are female, the gender breakdown of students participating in the literacy surveys illustrates that more females (almost 70 percent) took the online pre–course literacy test compared to the face–to–face classes where female students comprised 42 percent of the students taking the pre–course literacy exam. In addition, in both formats, approximately a third of the students had not taken a high school or college level economics course prior to beginning their economics course. The initial test, conducted during the first two weeks of a semester, will determine whether online and face–to–face groups are statistically similar (no difference between the means); if this is the case, a second test will be conducted to determine whether a statistically significant difference exists between the groups on the post–test.

This is a causal–comparative study to determine if students who take a course online differ when compared in economic literacy with students who take a course in the traditional face–to–face format before they complete a course in the principles of economics. In addition, whether there are any difference between students taking the course online and in the face–to–face mode exist after completion of the course will be tested.

The students who participated in the experiment were told to take the literacy test online and outside of class within two weeks of the beginning of class and were not told that they would be taking the test again at the end of the course. The students were assured that their responses would remain anonymous. The face–to–face principles of economics courses and the online courses were very similar in composition as far as books used, chapters covered, videos and PowerPoint presentations viewed, etc. The courses were also taught by the same instructor who had 14 years of full–time economics teaching experience, including six years of online teaching experience. Many lectures from the face–to–face classes were videotaped, digitalized and streamed so that students who were taking the class online could view the same lectures by the same professor. Discussions which were held in the face–to–face classes were simulated through asynchronous bulletin board postings.

Four different categories of variables have been collected over two semesters of students taking the principles of economics online and in the traditional face–to–face mode. Students in both modalities take an online economics literacy test, giving the variables pre–course online course literacy and pre–course face–to–face literacy. At the end of the course, the variables will be post course online and post face–to–face results. In both comparisons, the face–to–face variables will be the independent variable and the online variables will be the dependent variable.

In the pre–course literacy tests, there were 107 students who participated in the face–to–face courses and 59 students in the online courses; in the post–test, after student withdrawal and failure in participating in taking of the post–test, these numbers were 59 and 32, respectively. In order to ensure participation and anonymity, individual responses were not tracked and the pre– and post–test samples were not matched.

The instrument used was The Standards in Economics Survey developed by the National Council of Economic Education. The survey consists of twenty questions that test basic knowledge of economic concepts (see Appendix). The online version of this test has been taken over 31,000 times with an average score of 80 percent (National Council on Economic Education, 2005). This test’s reliability and validity have been proven in over 10 years of use and on thousands of respondents.

The data was analyzed using the Independent–Samples t Test with the underlying assumptions that the populations from which the samples are derived are normal, have equal variances and that the cases of online versus face–to–face students represent a random sample which are independent from each other (Green and Salkind, 2005).

The hypotheses to be tested are:
H1: No statistically significant difference between face–to–face and online students on economic literacy tests taken during the first two weeks of the semester; and,
H2: No statistically significant difference exists between online and face–to–face in economic literacy at the end of the course will also be tested.




At the 95 percent confidence level with alpha equal to .05 and a two–tailed test, there is not a statistically significant difference between the pre–course literacy scores on the test of economic literacy between the face–to–face students and the online students. At the end of the course, we cannot retain the hypothesis that students at the end of the course have similar results in economic literacy independent of whether the course is taken online or face–to–face.

Descriptive Statistics
The descriptive statistics (see Table 1) illustrate that the groups were similar in mean and standard deviation both initially and after completion of the courses. In a boxplot graph of the pre–literacy results and post–course literacy results, it was illustrated that at the beginning of the semester, the means of the two groups of students were very similar as were the ranges of scores; at the end of the course, the difference in the ranges and in the middle 50 percent of the scores appeared somewhat different. A key question is whether this difference is statistically significant.

Table 1: Descriptive statistics
  Number (n) of observations Mean (M) Standard deviation (STD)
Pre Ft2F
Pre Online
Post Ft2F
Post Online

Hypothesis: H1
Treating each class as an independent sample, the t value was calculated and compared to a critical t value. At the 95 percent confidence level with 164 (n-2) degrees of freedom and alpha equal to .05, given a two–tailed test, the critical t value of a two–tailed test would be 1.96 [3]. In the pre–test (H1), with equal variances assumed, the computed t ratio equals .208. This number is clearly in the region of the null hypothesis that there is no difference between the pre–literacy face–to–face and online students.

Hypothesis: H2
The results of the post–test (H2), tell a different story than that of the pre–course literacy results. At the 95 percent confidence level with 164 (n-2) degrees of freedom and alpha equal to .05., given a two–tailed test, the critical t value of a two–tailed test would be 2.0 [4]. The calculated t ratio is 2.2, in the region leading to the rejection of the null hypothesis. There is a slight, statistically significant difference between students taking a course in the principles of economics in the traditional face–to–face mode and online on the level of economic literacy attained as measured by the instrument, The Standards in Economics Survey.



Discussion and conclusion

Is student achievement in economic literacy after taking a course in the principles of economics dependent upon whether the course was taken face–to–face or online? Research in other subjects has shown that student achievement is independent of whether the course is taken in a face–to–face format or online leading to the no significant difference phenomenon (Allen, 2003; Laby and Comm, 2003; Russell, 1999 and 2003; Wang and Newlin, 2002).

While students may have a preference for taking economics in a face–to–face modality, this study illustrates that the format may matter when it comes to performance in economic literacy after taking a course in the principles of economics. Further research using larger populations and different instructors should be undertaken to investigate whether instructors of the principles of economics courses should actually be concerned that student achievement will be negatively impacted based solely upon the mode of delivery. In addition, the impact on gender must be investigated, namely why there were more females in the online classes and whether this had an impact on whether a significant difference existed between the face–to–face and online courses. End of article




The Standards in Economics Survey
Answer each of the 20 questions to the best of your ability.
1. A large increase in the number of fast–food restaurants in a community is most likely to result in:
A. Lower prices and higher quality.
B. Lower prices and lower quality.
C. Higher prices and higher quality.
D. Don’t know.
2. A person who starts a business to produce a new product in the marketplace is known as:
A. A manager.
B. A bureaucrat.
C. An entrepreneur.
D. Don’t know.
3. An increase from 5% to 8% in the interest rates charged by banks would most likely encourage:
A. Businesses to invest.
B. People to purchase housing.
C. People to save money.
D. Don’t know.
4. For most people, the largest portion of their personal income comes from:
A. Wages and salaries from their jobs.
B. Interest from stocks and bonds they own.
C. Rent paid to them from property they own.
D. Don’t know.
5. If the real gross domestic product of a country has increased, but the production of goods has remained the same, then the production of services has:
A. Increased.
B. Decreased.
C. Remained the same.
D. Don’t know.
6. If the price of beef doubled and the price of poultry stayed the same, people would most likely buy:
A. More poultry and less beef.
B. Less poultry and more beef.
C. The same amount of poultry and beef.
D. Don’t know.
7. If Macroland (a fictional country) stopped importing automobiles from Country X, who would be most likely to benefit?
A. Automobile manufacturers in Country X
B. Consumers in Macroland
C. Automobile manufacturers in the Macroland
D. Don’t know.
8. If your city government sets a maximum amount landlords can charge in rent, what is the most likely result?
A. There will be more apartments available than people want to rent.
B. There will be fewer apartments available than people want to rent.
C. The number of apartments available will be equal to the number of people that want to rent.
D. Don’t know.
9. In a market economy, who determines what goods and services should be produced?
A. Producers and government
B. Consumers and government
C. Producers, consumers, and government
D. Don’t know.
10. Mexico grows fruits and vegetables and Argentina produces beef. If Mexico voluntarily trades fruits and vegetables in exchange for Argentinean beef,
A. Both Mexico and Argentina benefit from the trade.
B. Both Mexico and Argentina lose from the trade.
C. Mexico benefits and Argentina loses from the trade.
D. Don’t know.
11. The manufacturers of XYZ winter sportswear have their manufacturing plants running night and day, but they are unable to produce enough sportswear to satisfy demand. If XYZ manufacturers cannot increase production and demand continues to increase, the price of XYZ winter sportswear will:
A. Increase.
B. Decrease.
C. Stay the same.
D. Don’t know.
12. The resources used in the production of goods and services are limited, so society must:
A. Make choices about how to use resources.
B. Try to obtain additional resources.
C. Reduce their use of resources.
D. Don’t know.
13. The stock market is an example of an institution within our economy that exists to help people achieve their economic goals. The existence of this institution:
A. Results in an increase in the price of stocks.
B. Brings people who want to buy stocks together with those who want to sell stocks.
C. Helps predict stock earnings.
D. Don’t know.
14. When a person rents an apartment, who benefits from the transaction?
A. Only the person renting the apartment.
B. Only the landlord.
C. Both the person renting the apartment and the landlord.
D. Don’t know.
15. When deciding which of the two items to purchase, one should always:
A. Choose the item that costs less.
B. Choose the item with the greatest benefits.
C. Choose an item after comparing the costs and benefits of both items.
D. Don’t know.
16.When governments supply products and services, these products and services usually benefit:
A. More than one person at a time whether they have paid for it or not.
B. Only the people who pay for these products and services.
C. Business at the expense of consumers.
D. Don’t know.
17.When a government’s expenditures for a year are greater than its revenue for that year, the difference is known as:
A. A balanced budget.
B. A budget deficit.
C. A budget surplus.
D. Don’t know.
18.Which of the following are most likely to be helped by inflation?
A. People living on a fixed income.
B. Banks that loaned money at a fixed rate of interest.
C. People who borrowed money at a fixed rate of interest.
D. Don’t know.
19.Which of the following would be most likely to accelerate innovation in the computer industry?
A. Placing a tax on all new inventions in the computer industry.
B. Increasing government regulation of the computer industry.
C. Investing in more research and development in the computer industry.
D. Don’t know.
20.Which one of the following statements about the function of money is wrong?
A. Money makes it easier to save.
B. Money makes trading goods and services easier.
C. Money holds its value well in times of inflation.
D. Don’t know.


About the author

Frank Albritton is a teacher with 15 years of full–time teaching experience at colleges and universities, and an author, specializing in productivity improvement and the economics of the health care sector. In 1998, and again in 2005, he was recognized for his excellence in teaching by being selected to be included in the “Who's Who Among America’s Teachers.” He is also a recognized leader in online economics education, receiving an award at the Eleventh International Conference on College Teaching & Learning for "Innovative Excellence in Teaching, Learning and Technology”.



1. Rader, 1996, p. 5.

2. Stern (2002) in a speech at the annual meeting of the Virginia Council on Economic Education.

3. Coladarci, et al., 2004, p. 438.

4. Ibid.



I.E. Allen and J. Seaman, 2003. Sizing the opportunity: The quality and extent of online education in the United States, 2002 and 2003.. Bedham, Mass.: Sloan Consortium, at

M. Allen, J. Bourhis, N. Burrell, and E. Mabry, 2002. “Comparing student satisfaction with distance education to traditional classrooms in higher education: A meta–analysis,” American Journal of Distance Education, volume 16, number 2, pp. 83–97.

L. Bertagnoli, 2001. “Education reservation,” Marketing News, volume 35, number 4 (12 February), p. 5.

T.J. DeLoughry, 1995. “Will higher education thrive or wither in cyberspace? High level administrators meet to debate on future of university higher education,” Chronicle of Higher Education, volume 41, number 20, p. A22 (27 January).

J. Gleason and L.J. van Scyoc, 1995. “A report on the economic literacy of adults,” Journal of Economic Education, volume 26, number 3, pp. 203–210.

S.B. Green and N.J. Salkind, 2005. Using SPSS for Windows and Macintosh: Analyzing and understanding data. Fourth edition. Upper Saddle River, N.J.: Pearson/Prentice Hall.

W.L. Hansen, M.K. Salemi, and J.J. Siegfried, 2002. “Use it or lose it: Teaching literacy in the economics principles courses,” American Economic Review, volume 92, number 2, pp. 463–472.

D.G. Labay and C.L. Comm, 2003. “A case study using gap analysis to assess distance learning versus traditional course delivery,” International Journal of Educational Management, volume 17, number 7, pp. 312–317.

National Council on Economic Education. 2005. “Campaign for economic literacy: Test your own economic literacy,” at

W.D. Rader, 1996. “Toward a philosophy of economics education,” Social Studies, volume 87, number 1, pp. 4–7.

T.L. Russell (compiler), 1999. The no significant difference phenomenon: As reported in 355 research reports, summaries and papers. Raleigh, N.C.: North Carolina State University.

T.L. Russell (compiler), 2003. “The no significant difference phenomenon,” at

G.H. Stern, 2002. “Top of the ninth: Economic literacy leads to better grasp of public policy issues,” The Region (Federal Reserve Bank of Minneapolis), at

W. Walstad and S. Allgood, 1999. “What do college seniors know about economics?” American Economic Review, volume 89, number 2, pp. 350–354. .

A.Y. Wang and M.H. Newlin, 2002. “Predictors of Web–student performance: The role of self–efficacy and reasons for taking an on–line class,” Computers in Human Behavior, volume 18, number 2, pp. 151–163.

W.C. Wood and J.M. Doyle, 2002. “Economic literacy among corporate employees,” Journal of Economic Education, volume 33, number 3, pp. 195–205.

A.D. Zimmerman, 2004. “Gauging the development of economic literacy in an introductory undergraduate economics course,” at


Editorial history

Paper received 24 October 2005; revised 3 May 2006; accepted 18 July 2006.

Contents Index

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.

Performance on tests of economic literacy: A comparison of face–to–face with online instruction by Frank P. Albritton, Jr.
First Monday, volume 11, number 10 (October 2006),

A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2017. ISSN 1396-0466.