Estimating the Effect of a Teacher Training Program on Advanced Placement® Outcomes


  • Richard Brown West Coast Analytics
  • Emily Anne Brown University of North Texas


computer science, professional development, teacher training


This study employs a potential outcomes modeling approach to estimate the causal effect of’s Professional Learning Program on Advanced Placement (AP) Computer Science Principles test taking and qualifying score earned for a recent cohort of 167 schools compared to a matched group of comparison schools. Results indicate substantial and significant increases in both Computer Science AP test taking and qualifying score earning for all students. In addition, the significant effects were even greater for Computer Science AP test taking and qualifying score earned by female and minority students when impact ratios are analyzed separately. This study provides evidence of a teacher training program that is having a significant and important impact on preparing more students to succeed in computer science and improve the future of computer science education in this country.

Keywords: computer science, professional development, teacher training


Download data is not yet available.

Author Biography

Emily Anne Brown, University of North Texas

Educational Psychology

Graduate Student


Austin, P. C. (2008). A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Statistics in Medicine, 27, 2037-2049. DOI:

Beede, D. N., Julian, T. A., Langdon, D., McKittrick, G., Khan, B., and Doms, M. E., Women in STEM: A Gender Gap to Innovation (August 1, 2011). Economics and Statistics Administration Issue Brief No. 04-11. Available at SSRN: or DOI:

Dawid, A. P. (2000). Causal inference without counterfactuals. Journal of the American Statistical Association, 95(450), 407-424. DOI:

Dougherty, C., Mellor, L. & Jian, S. (2006). The relationship between Advanced Placement and college graduation. (National Center for Educational Accountability: 2005 AP Study Series, Report 1). Austin, TX: National Center for Educational Accountability.

Glass, T.A., Goodman, S.N., Hernan, M.A., & Samet, J.M. (2013). Causal inference in public health. Annual Review of Public Health, 34, 61-75. DOI:

Goode, J. (2007). If You Build Teachers, Will Students Come? The Role of Teachers in Broadening Computer Science Learning for Urban Youth. Journal of Educational Computing Research, 36(1), 65-88. DOI:

Hargrove, L., Godin, D., Dodd, B. (2008). College outcomes comparisons by AP and non-AP high school experiences (College Board Research Report 2008-3). New York: The College Board.

Holland, P. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945-960. DOI:

Imai, K., & Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society: Series B. 53, 597-610.

Keele, L. (2015). The statistics of causal inference: A view from political methodology. Political Analysis, 23, 313-335. DOI:

Louckes-Horsely, S., Stiles, K.E., Mundry, S., Love, N., Hewson, P.W. (2010). Designing Professional Development for Teachers of Science and Mathematics (3rd edition). 69-70.

Mattern, K.D., Marini, J.P., & Shaw, E.J. (2013). The relationship between AP Exam performance and college outcomes. (College Board Research Report 2009-4) New York: The College Board.

Mattern, K.D., Shaw, E.J., & Xiong, X. (2009). Are AP students more likely to graduate from college on time? (College Board Research Report 2013-5) New York: The College Board.

McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9, 403-425. DOI:

Morgan, R., & Klaric, J. (2007). AP students in college: An analysis of five-year academic careers (College Board Research Report No. 2007-04). New York: The College Board.

Mouza, C., Marzocchi, A, Pan, Y., & Pollock, L. (2016). Development, Implementation, and Outcomes of an Equitable Computer Science After-School Program: Findings From Middle-School Students. Journal of Research on Technology in Education. 48. 1-21. 10.1080/15391523.2016.1146561. DOI:

Murphy, D., & Dodd, B. (2009). A comparison of college performance of matched AP and non-AP student groups. (College Board Research Report No. 2009-6). New York: The College Board.

Pearl, J. (2009). Causality: models, reasoning, and inference. 2nd Edition. New York: Cambridge University Press. DOI:

Ridgeway, G., McCaffrey, D., Morral, A., Burgette, L, & Griffin, B. (2015). “twang: Toolkit for weighting and analysis of nonequivalent groups.†Available at

Rosenbaum, P. R. (2002). Observational Studies, 2nd ed. Springer, New York.

Rosenbaum, P.R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33-38.

Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469), 322-331. DOI:

Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistical Medicine, 26, 20-36. DOI:

Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores, relating theory to practice. Biometrics, 52, 249-264. DOI:

Sax, L. J., Lehman, K. J., Jacobs, J. A., Kanny, M. A., Lim, G., Monje-Paulson, L., & Zimmerman, H. B. (2017). Anatomy of an Enduring Gender Gap: The Evolution of Women’s Participation in Computer Science, The Journal of Higher Education, 88:2, 258-293, DOI: 10.1080/00221546.2016.1257306 DOI:

Shaw, E. J., Marini, J. P., & Mattern, K.D. (2013). Exploring the utility of Advanced Placement participation and performance in college admission decisions. Educational and Psychological Measurement, 73, 229-253. DOI:

Stuart, E. A. (2010). Matching methods for causal inference: a review and a look forward. Statistical Science, 25(1), 1-21. DOI:

West, S. G., & Thoemmes, F. (2010). Campbell’s and Rubin’s perspectives on causal inference. Psychological Methods, 15(1), 18-37. DOI:

Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved from



How to Cite

Brown, R., & Brown, E. A. (2019). Estimating the Effect of a Teacher Training Program on Advanced Placement® Outcomes. International Journal of Computer Science Education in Schools, 2(4), 3–21.