Developing and Assessing Computational Thinking in Secondary Education using a TPACK Guided Scratch Visual Execution Environment

Keywords: TPACK;, Computational Thinking, Assessment, VEE, Scratch


Effective and reliable assessment approaches to computational thinking in secondary education are in demand. This paper uses a guided technological pedagogical content knowledge (TPACK) framework, incorporating a visual execution environment (VEE) and Scratch project for secondary school students as a method to teach and assess computational thinking. The objective is to investigate if computational thinking and programming concepts can be improved upon following this method, and if the K-12 children are able to improve their computational thinking skills. The research study was conducted over 2 years in a school setting using the guided VEE and project developed following the dimensions of Computational Thinking process. The project participants came from two cohorts, an after-school programming camp and an in-school environment. Data was collected over two academic years and a quasi-experimental procedure with pre- and post-test was followed. The results demonstrate knowledge gain on computational and programming concepts and encourages us to convey how students translate (as opposed to transfer) their computational thinking experiences into reality. The results indicate the students achieved significant improvement in their computational thinking development.


Download data is not yet available.


Aho, A. V. (2012). Computation and computational thinking. The Computer Journal, 55(7), 832-835.
Álvarez Martínez, C., & Llosa Espuny, J. (2010). Formative evaluation with quick feedback using interactive controls. Paper presented at the XVI Conference on University Teaching of Computing, University of Santiago de Compostela. Escola Técnica Superior d'Enxeñaría.
Armoni, M., Meerbaum-Salant, O., & Ben-Ari, M. (2015). From scratch to “real” programming. ACM Transactions on Computing Education (TOCE), 14(4), 1-15.
Baron, G. L., Drot-Delange, B., Grandbastien, M., & Tort, F. (2014). Computer science education in French secondary schools: Historical and didactical perspectives. ACM Transactions on Computing Education (TOCE), 14(2), 1-27.
Basu, S., Kinnebrew, J. S., & Biswas, G. (2014). Assessing student performance in a computational-thinking based science learning environment. In International conference on intelligent tutoring systems (pp. 476-481): Springer.
Bell, T., Andreae, P., & Lambert, L. (2010). Computer science in New Zealand high schools. Paper presented at the 12th Australasian Conference on Computing Education, Brisbane, Australia.
Bell, T., Andreae, P., & Robins, A. (2012). Computer science in NZ high schools: the first year of the new standards. . Paper presented at the 43rd ACM technical symposium on Computer Science Education.
Bell, T., Andreae, P., & Robins, A. (2014). A case study of the introduction of computer science in NZ schools. ACM Transactions on Computing Education (TOCE), 14(2), 1-31.
Bell, T., Duncan, C., & Atlas, J. (2016). Teacher feedback on delivering computational thinking in primary school. Paper presented at the 11th Workshop in Primary and Secondary Computing Education.
Bienkowski, M., Snow, E., Rutstein, D., & Grover, S. (2015). Assessment design patterns for computational thinking practices in secondary computer science: A first look. Retrieved from SRI Education.
BOE. (2015). Orden ECD/65/2015 por la que se describen las relaciones entre las competencias, los contenidos y los criterios de evaluación de la Educación Primaria, la Educación Secundaria Obligatoria y el Bachillerat. Retrieved from
Brennan, K., Balch, C., & Chung, M. (2014). Creative computing: Scratch curriculum guide. Retrieved from
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. Paper presented at the American educational research association, Vancouver, Canada.
Brown, N. C., Kölling, M., Crick, T., Peyton Jones, S., Humphreys, S., & Sentance, S. (2013). Bringing computer science back into schools: Lessons from the UK. Paper presented at the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). New York.
Brown, N. C., Sentance, S., Crick, T., & Humphreys, S. (2014). Restart: The resurgence of computer science in UK schools. ACM Transactions on Computing Education (TOCE), 14(2), 9.
Cohen, J. (1988). Statistical Power Analysis for the Behavioural Sciences. Hillsdale, NJ, USA: Erlbaum.
Collective, B. M., & Shaw, D. (2012). Makey Makey: improvising tangible and nature-based user interfaces. Paper presented at the Sixth International Conference on Tangible, Embedded and Embodied Iinteraction.
Deek, F. P., & Kimmel, H. (1999). Status of computer science education in secondary schools: One state's perspective. Computer Science Education, 9(2), 89-113.
Duncan, C., Bell, T., & Atlas, J. (2017). What do the teachers think? Introducing computational thinking in the primary school curriculum. Paper presented at the Nineteenth Australasian Computing Education Conference.
Fields, D. A., Searle, K. A., Kafai, Y. B., & Min, H. S. (2012). Debuggems to assess student learning in e-textiles. Paper presented at the 43rd ACM technical symposium on Computer Science Education.
García-Peñalvo, F. J., & Mendes, A. J. (2018). Exploring the computational thinking effects in pre-university education. Computers in Human Behavior, 80, 407-411.
Granic, I., Lobel, A., & Engels, R. C. (2014). The benefits of playing video games. American psychologist, 69(1), 66.
Grover, S. (2011). Robotics and engineering for middle and high school students to develop computational thinking. Paper presented at the American Educational Research Association, New Orleans, LA.
Grover, S., & Pea, R. (2013). Computational Thinking in K–12: A Review of the State of the Field. Educational Researcher, 42(2), 38-43.
Hargreaves, A., Earl, L. M., & Ryan, J. (1996). Schooling for change: Reinventing education for early adolescents: Routledge.
Hemmendinger, D. (2010). A Plea for Modesty. ACM Inroads, 1(2), 4-7.
Hernández Tijera, I., & Perianes Rodriguez, B. (2018). Camino hacia la excelencia. Premios Trabajo Fin de Máster en Formación del profesorado de Educación Secundaria. Retrieved from Colegio Oficial de Docentes. Sial Pigmalión:
Hubwieser, P. (2012). Computer science education in secondary schools: The introduction of a new compulsory subject. Transactions in Computing Education, 12(4), 161-164.
INTEF. (2019). Programación, robótica y pensamiento computacional en el aula: Situación en España.
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology & Teacher Education, 9(1), 60-70.
Koh, K. H., Basawapatna, A., Bennett, V., & Repenning, A. (2010). Towards the automatic recognition of computational thinking for adaptive visual language learning. IEEE Symposium on Visual Languages and Human-Centric Computing, 59-66.
Koops, W. R. (2012). Johan Huizinga 1872–1972: Papers Delivered to the Johan Huizinga Conference Groningen 11–15 December 1972. Springer.
Lee, E., Kafai, Y. B., Vasudevan, V., & Davis, R. L. (2014). Playing in the arcade: Designing tangible interfaces with MaKey MaKey for Scratch games. In Playful user interfaces (pp. 277-292). Singapore: Springer.
Levy, R. B. B., Ben-Ari, M., & Uronen, P. A. (2003). The Jeliot 2000 program animation system. Computers & Education, 40(1), 1-15.
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior.
Malone, R. (2011). Curriculum Studies. In B. Walsh (Ed.), Education Studies in Ireland: the Key Disciplines. Dublin: Gill & Macmillan Ltd.
Martinez, S. L., & Stager, G. (2013). Invent to learn. Making, Tinkering, and Engineering in the Classroom. Paper presented at the Construting Modern Knowledge., Torrance, Canada.
Moreno-León, J., & Robles, G. (2015). Dr. Scratch: A web tool to automatically evaluate Scratch projects. Paper presented at the Workshop in primary and secondary computing education.
Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking. Revista de Educación a Distancia, 46, 1-23.
NCCA. (2017). Leaving Certificate Computer Science Specification. Retrieved from
NRC. (2010). Report of a Workshop on the Scope and Nature of Computational Thinking National Research Council, Washington DC: National Academies Press
Piaget, J., & Inhelder, B. (1999). The Child's Conception of Space. UK: Routledge.
Resnick, M., & Robinson, K. (2017). Lifelong kindergarten: Cultivating creativity through projects, passion, peers, and play: MIT Press.
Román-González, M., Pérez-González, J. C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior, 72, 678-691.
Sáez-López, J. M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 128-141.
Snow, E., Tate, C., Rutstein, D., & Bienkowski, M. (2017). Assessment design patterns for computational thinking practices in exploring computer science. Retrieved from SRI Education.
Urbina Ramírez, S. (1999). Informática y teorías del aprendizaje. Píxel-Bit. Revista de medios y educación, 12, 87-100.
Velázquez-Iturbide, J. Á. (2018). Report of the Spanish Computing Scientific Society on Computing Education in Pre-University Stages. Paper presented at the TEEM'18: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, Salamanca, 2018.
Walsh, B., & Dolan, R. (2009). A guide to teaching practice in Ireland: Gill & Macmillan Ltd.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127-147.
Weintrop, D., & Wilensky, U. (2017). Comparing block-based and text-based programming in high school computer science classrooms. ACM Transactions on Computing Education (TOCE), 18(1), 1-25.
Werner, L., Denner, J., Campe, S., & Kawamoto, D. C. (2012). The fairy performance assessment: measuring computational thinking in middle school. Paper presented at the 43rd ACM technical symposium on Computer Science Education.
Wing, J. M. (2006). Computational Thinking. Communications of the ACM, 49(3).
Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717-3725.
Wing, J. M. (2011). Computational thinking. Paper presented at the 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
Yadav, A., Gretter, S., Good, J., & McLean, T. (2017). Emerging Research, Practice, and Policy on Computational Thinking. In R. P. & H. C. (Eds.), Emerging Research, Practice, and Policy on Computational Thinking. Educational Communications and Technology: Issues and Innovations (pp. 205-220): Springer, Cham.
Yadav, A., Gretter, S., Hambrusch, S., & Sands, P. (2016). Expanding computer science education in schools: understanding teacher experiences and challenges. Computer Science Education, 26(4), 235-254.
How to Cite
Hijón Neira, R., Garcia-Iruela, M., & Connolly, C. (2021). Developing and Assessing Computational Thinking in Secondary Education using a TPACK Guided Scratch Visual Execution Environment. International Journal of Computer Science Education in Schools, 4(4), 3-23.