Culturally relevant and responsive pedagogy in computing: A Quick Scoping Review

The underrepresentation of certain groups in computing has led to increasing efforts to develop computing curricula that is responsive and relevant to a more diverse group of learners. The current paper used a Quick Scoping Review methodology to identify research that has implemented and evaluated culturally responsive and relevant K-12 computing curricula, and to understand how they have been designed, the methods used for evaluation, and the factors affecting their success. In total, 12 papers were included in the review, and all were from a United States setting. Successes included changing learners’ attitudes towards computing and increased learning gains. Key factors in the implementation of the curricula were teacher confidence and understanding of the socio-political context of computing, opportunities provided for collaboration and sharing knowledge and opinions, and allowing time for difficult discussions without oversimplifying the issues. The review identifies important lessons to be learned for educators around the world who are aiming to increase diversity in representation in computing in their schools.


Introduction
Recently, increasing attention has been paid to the cultural relevance of computing curricula. In this paper, we aim to highlight key areas of research and practice and identify factors influencing the success of interventions focused on underrepresented ethnic and cultural groups in computing. We will begin by providing an overview of key theories underpinning the work on cultural responsiveness in teaching. We will then review research focusing on the design and evaluation of computing resources that have culture, equity, and social justice as their focus. Finally, we will consider the implications of the research for the future development of computing curricula.

Cultural capital and Critical Race Theory
Cultural capital is an important, and often neglected, consideration when developing curricula or resources in education. The term refers to internal aspects of individuals that they share with members of their families and their communities, such as language, knowledge, and belief systems, but also to external products of culture, such as artistic expression (Bourdieu, 1986). Yosso (2005) has argued that the culture of dominant groups is that which is most valued in society and becomes the 'norm' by which other cultures or groups are judged.
According to Yosso (2005;p.75), although educational institutions may aim to provide neutral settings for learning, in reality they tend to promote standards and topics valued by the dominant groups in society, often resulting in "deficit thinking". This results in students from non-dominant, or minority, groups being viewed as problems to be fixed, and who need to learn the dominant group's cultures and ways in order to succeed in life (Cabrera, 2019). Several authors have highlighted how Critical Race Theory can help to identify, examine and challenge these deficit models, drawing from law, history and social theory to bring attention to how race and racism can implicitly bias and impact educational practices (e.g., Ladson-Billings & Tate, 1995;Lynn & Dixson, 2013;Yosso, 2005).
For example, having a computer at home allows a young person to develop computing skills and vocabulary, which is cultural capital that is valued in the school environment. Other children may have developed skills which are of great value in their culture but are not those that are valued by the dominant groups (Yosso, 2005). Thus, children from minority backgrounds can often be labelled sub-standard or deficient in certain 'basic' skills that are based on norms of middle-class white children which have been assumed to be applicable to children from all backgrounds (Lachney, 2017). Understanding these implicit assumptions and being open to identifying and supporting minority groups' cultural capital is of great importance in the education system. One step towards this goal is through the development of more culturally relevant and accessible teaching approaches and resources for young people from different backgrounds, on which we will focus for the rest of this paper.

Developing a theory addressing culture and equity in computing education
Theories of equitable teaching and learning practices which focus on cultural relevance and responsiveness have developed over several decades, leading to frameworks such as Culturally Relevant Pedagogy (Ladson-Billings, 1995), Culturally Responsive Teaching (Gay, 2000), and Culturally Sustaining Pedagogy (Paris, 2012;see Madkins et al., 2020, for an overview of these approaches). The key elements of these frameworks all focus on how teachers understand, respond to and use the cultural diversity within their communities to help all students achieve, build relationships, celebrate and sustain their cultures, and understand and challenge inequitable practices and belief systems that marginalise minority groups.
Building on these frameworks in education, colleagues (2013, 2015) developed Culturally Responsive Computing (CRC) theory to focus on equitable teaching approaches in computing specifically. Through an extracurricular club for girls from minority groups (COMPUGIRLS) that was held over a period of two years, the authors promoted three main tenets of an equitable teaching approach: asset building, reflection, and connectedness. To address these tenets, the teacher/mentors were encouraged to identify and integrate the girls' technological and subject area knowledge into the curriculum, to support the girls in reflecting on their own knowledge and how it had developed, and to explicitly make links with their peers and communities, and wider cultural/socio-political issues (Scott & White, 2013). After implementing the COMPUGIRLS program with a range of different groups, Scott et al. (2015) revised the theory to incorporate more nuance, resulting in the following five principles: 1) All students are capable of digital innovation 2) The learning context supports transformational use of technology 3) Learning about one's self along various intersecting sociocultural lines allows for technical innovation 4) Technology should be a vehicle by which students reflect and demonstrate understanding of their intersectional identities 5) Barometers for technological success should consider who creates, for whom, and to what ends, rather than who endures socially and culturally irrelevant curriculum. Scott et al. (2015;pp. 420-421) These theoretical principles move away from a deficit model of thinking, in which the focus of computing teaching for minority groups is decontextualised 'basic' skills that they are seen to lack, and allows students to express their identities and their culture through technology in a way that is meaningful to them and their communities (Scott et al., 2015). They provide authentic learning experiences, in which students experience teaching and learning that is both interesting and relevant to them ("personal authenticity") and which reflects ways of working within computer science that will be of use in their future careers ("professional authenticity"; Means & Stephens, 2021, pp.19-20).
The theory also promotes a critical engagement with technology and social justice issues to allow students to innovate and create technological solutions to address issues that affect them (Madkins et al., 2020). It allows students who have a focus on communal goals (i.e., those that involve working with, or for the good of, others; Brinkman & Diekman, 2016) to identify computing as a means of addressing these goals and, therefore, to feel a sense of belonging within the subject. Given that students from minority groups may have a greater focus on communal goals than those from dominant groups (Lewis et al., 2019), it is of great importance that computing as a discipline is seen to align with these goals in order to encourage more diversity in those choosing the subject and career.
Although based on implementation in an extracurricular setting, the COMPUGIRLS program and CRC theory have important applications for teaching computing in both formal and non-formal learning environments. The next section provides an overview of a range of curricula that have been developed for use in both environments that are focused on culture and equity, and that incorporate many of the principles of CRC theory.

Teaching computing through the lens of culture and equity
Over the last 20 years, researchers and practitioners have increasingly aimed to develop computing curricula that are culturally responsive and relevant. Designing equitable and authentic learning experiences in computing requires a conscious effort to take into account characteristics of learners and their social environments, and to deal with topics that are relevant to a wide range of students.
Means and Stephens (2021) outline three key areas on which educators should focus when developing authentic computing experiences: the learners (understanding who they are and the experiences they bring to the computing classroom), the community (recognising and understanding the knowledge, opinions and experiences of both the local and the learning community, and building on the cultural wealth in these communities), and the activities (identifying both personally and professionally meaningful computing tasks, as well as considering meaningful learning outcomes and how they are valued). Madkins et al. (2020) describe three overlapping but separate equitable teaching practices in computing: promoting identity development by understanding the learners but also by allowing them to express themselves through computing; highlighting the personal and sociopolitical relevance of technology by situating technology ideas within their local community and wider sociopolitical context and allowing learners to address issues that are meaningful to them; and positioning learners as creative agents/change agents by empowering them to use technology to innovate and solve problems with personal relevance.
Both sets of principles suggest that students should not only be given the opportunity to express their cultural knowledge and identities in computing tasks, but also to develop a sense of identity and belonging within computing as a discipline and a profession. This involves challenging stereotypes of who can be a computer scientist and what computing should be used to achieve in society, as well as understanding and addressing power relationships and biases within the community of learners. Learners should feel empowered to be creative and work with others to innovate and solve problems important to them, their communities, and wider society.

The current paper
The current paper aimed to identify key approaches to embedding culturally responsive and relevant pedagogy into computing curricula and to understand the design principles that were used in the development of the curricula. Specifically, we were interested in approaches that had been used in formal and non-formal learning spaces with K-12 students and had been evaluated in terms of student outcomes. We utilised a Quick Scoping Review (QSR) methodology (Collins et al., 2015) to allow us to synthesise the evidence available and provide an informed conclusion concerning the previous literature. Although less rigorous than a full systematic review or meta-analysis, the QSR method remains transparent and minimises bias while allowing the author to answer more open-ended questions about the evidence on a subject (Collins et al., 2015). For the current paper, we aimed to address the following questions: 1) Which design principles have been embedded into the curricula to promote cultural relevance, equity and justice in computing? 2) How has the success of the teaching approaches or curricula been evaluated? 3) Which factors positively or negatively influence the success of the teaching approaches or curricula?

Initial search
Our investigation of the literature took place between January and March 2021. We began by searching for a recent review paper on culturally relevant pedagogy in computing through Google Scholar, using the term "culturally relevant computing review". This identified a narrative review conducted by Morales-Chicas et al.
(2019) which included 22 papers from JSTOR Arts and Sciences, Web of Science, and ERIC databases on three key themes that had been identified in a previous scoping exercise, and which were used in the systematic literature search: culturally responsive computing, ethnocomputing, and Culturally Situated Design Tools. Ethnocomputing relies on the use of relevant cultural artifacts and symbols, as opposed to Western ones that pervade most computing curricula (Tedre et al., 2006). Culturally situated design tools (CSDTs) are developed to this end, producing visual programming media reflecting different cultural practices and artifacts, and codesigned with the community for whom the practice or artifact is relevant (Eglash et al., 2006). The term 'etextiles' had also been added to the search terms as these teaching activities often include elements of culturally relevant pedagogy, although they do not specifically adopt a culturally responsive approach (Morales-Chicas et al., 2019). Papers were included in the review if they were written in English in peer-reviewed journals between 1998-2018, and if they focused on K-12 education.

Initial review
We read the review paper by Morales-Chicas et al. (2019), along with the 22 papers included in the review. For our current purposes, we decided to focus on studies evaluating outcomes for K-12 students receiving culturally relevant and responsive curricula, in both formal and non-formal education settings. Evaluations included quantitative or qualitative measures of learning, attitudes, engagement, or other outcomes, and were gathered from both students and teachers/instructors. Papers must be written in English.
From the 22 papers, two position and two review papers were excluded, along with one paper focused on teacher professional development, and four that did not specifically aim to take a culturally responsive approach. Seven more papers that described culturally relevant approaches without full evaluation were also excluded, resulting in a total of 6 studies remaining in the current paper from the original systematic review.

Additional search, review, and analysis
Since the review had covered literature up to 2018, we conducted an additional search for papers using the same search terms as the original review between 2018 and March 2021. We added the search term "Exploring Computer Science" to identify papers evaluating the school-based curriculum developed for high school students using culturally relevant and equity-focused approaches (Goode, 2010). We replaced the Web of Science database, to which we did not have access, with the ACM Digital Library. From JSTOR Arts and Sciences, we retrieved 8 additional papers across the original search terms, but none of them met the inclusion criteria. In ERIC, a further 9 papers were retrieved, with one paper meeting our inclusion criteria. ACM Digital Library produced three papers for inclusion from 86 results. Two additional papers from 94 potential results using the "Exploring Computer Science" search term were also found across the three databases.
The final QSR therefore included 6 papers from the original systematic review by Morales-Chicas et al. (2019), and 6 additional papers published between 2018 and March 2021.

Results
The 12 studies emerging from the QSR are presented in Table 1 and are linked to key design principles recommended by Madkins et al. (2020) and Means and Stephens (2021).
In-class ECS curriculum over 1 year, using inquiry-based approaches, culturally relevant and culturally responsive pedagogy 70 high school students (US) Qazi et al. (2020) In-class ECS curriculum over 1 year, using inquiry-based approaches, culturally relevant and culturally responsive pedagogy 398 high school students

US = United States
Beginning with the review by Morales-Chicas et al. (2019), the length of the curricula ranged from three days to two years, with the majority of courses being delivered outside of school or in elective programs and with relatively low numbers of young people. Three studies utilised CSDTs, with results indicating improvements in students' attitudes towards computing/science (Eglash et al., 2011;Eglash et al., 2013), and/or in knowledge after lessons (Babbitt et al., 2015;Eglash et al., 2011), in comparison to groups not receiving the cultural curricula. Interestingly, Eglash et al. (2013) compared a heritage culture and a vernacular culture approach and reported a slight preference for vernacular culture amongst students. The authors point out that preferences and interests are highly likely to change at different points in young people's development, which suggests that using a range of different cultural touch points may therefore provide the best results. It is important to note that these quantitative evaluations of relatively short, focused interventions relied on very small sample sizes and further research will be required to better understand the impact of using CSDTs on a larger scale and in the longer term. Furthermore, more research is required to understand differences between formal and non-formal settings in terms of how these tools are integrated into the wider curriculum, how much time is allowed for their exploration and use, and their impact on computing-specific knowledge and skills.
Some studies used a more vernacular culture approach, using video games and digital storytelling to engage young people from underrepresented groups in computing. The COMPUGIRLS program (Ashcraft et al., 2017;Scott & White, 2013) incorporates three courses: digital storytelling, designing educational video games using Scratch, and designing projects in virtual worlds. Over the course of the programs, the authors' qualitative analyses reveal changes in the girls' understanding of their own identities and their roles within the community through computing, as well as the development of key computing skills. DiSalvo and colleagues (2014) drew on the popularity of video games to engage African American male students in computing through a program in which students became games testers and participated in computing workshops. The program was considered successful in improving interest and confidence in computing amongst the students, with 65 percent choosing to study computer science after high school. The non-formal settings of these programs, along with the engaging content, seem to provide a platform for young people who are typically underrepresented in computing to explore different aspects of the subject and understand the relevance of it to their lives and future careers. Trying to incorporate these types of approaches into the formal education system can be more of a challenge, although recent studies (outlined below) are attempting to do so. . We will discuss each of these sets of studies in turn. Davis et al. (2019) aimed to incorporate cultural computing into a standard Python programming course in schools and reported significant increases in students' learning of computing concepts during this time. Content that had taken nine months to learn in the standard course was covered in only six months in the adapted course. However, there was very little change in attitudes towards computing or evidence of increased understanding of the relationships between culture and computing as a result of the adapted curriculum. This perhaps suggests that there needs to be a very careful balance between the cultural and computing content in a curriculum, especially in formal schooling, to have an effect on both learning and cultural awareness or identity. Teachers who are delivering the curriculum have to ensure that there are clear learning gains from an activity which can be evidenced through formal testing and may have less time available to promote exploration and discussion than researchers or instructors delivering the same content in non-formal settings. However, repeating the study across different schools, teachers and students will be important in understanding the interplay between these factors outside of the single class of high school students involved in this particular study.
A greater focus on the computing content of the learning experience is clear in the two Scratch-based curricula designed for younger students in the studies by Yang et al. (2021) and Franklin et al. (2020). Yang et al. incorporated a culturally responsive computing element into a standard 'Use-Modify-Create' framework for teaching computing (Lee et al., 2011) in a non-formal setting. They highlighted opportunities in both the 'Modify' and 'Create' stages to develop a sense of identity and belonging and to produce personally relevant and meaningful artifacts through computing. In-depth case studies of two participants in the 11-week program revealed indicators of increased belonging to the community of their computing club, as well as clear progression in their understanding and implementation of computing concepts. However, further research is required with a much larger sample and with a greater focus on cultural identity outside of computing to evaluate this program's success.
The Scratch Encore curriculum (Franklin et al., 2020), on the other hand, has been piloted with 271 5th-8th Grade students in a school environment over the course of a year. It consists of sequential modules covering intermediate computing concepts, and each module is offered across three different strands: Multicultural, Youth Culture, and Gaming (all of which cover the same computing content but with different themes). The strands draw on both heritage and vernacular culture, and educators can choose which is the most relevant for their learners. Like the previous study, it utilises the Use-Modify-Create framework, providing opportunities for personalisation and expressing an individual's identity through the projects that are created. The pilot evaluation revealed that teachers found the course engaging for their students and the difficulty level appropriate. Opportunities for incorporating culture and students' own identities was demonstrated through the projects students created, and teachers' feedback on the relevance of different lessons. Further research is required to assess the learning gains of students, as well as changes in their attitudes and confidence, using this course over time. Direct comparisons of the curriculum in formal and non-formal settings will also provide some insight into the impact of the purpose of the activities, and the role of the instructor and their instruction approaches, on student outcomes.
Based entirely in formal education settings, three recent studies have evaluated the implementation of the ECS curriculum over five different states (McGee et al., 2018;Ryoo, 2019;Qazi et al., 2020). Student engagement and interest in CS were positively affected across all three studies, with Qazi et al. reporting students' perceived improvement specifically in creativity, problem solving, critical thinking and collaboration. These improvements were related to how engaged students felt in the curriculum, including how autonomous they felt in solving reallife problems. The importance of real-world problem-solving for student engagement was also evident in Ryoo's qualitative analyses, along with being able to demystify CS practices and careers, and demonstrations of how students' voices and perspectives were valued and could make a difference to others. As well as changes in attitudes and engagement in response to the ECS curriculum, McGee et al. revealed significant learning gains in computational thinking over the course of the year, irrespective of students' gender and race. However, it is to be noted that both students' end of course attitudes and the teachers' years of teaching ECS were significant predictors of these learning gains.
The impact of teachers' years of experience may be due to increasing familiarity with the course content and their pedagogical and concept knowledge, but is also likely to be related to their developing understanding of the equity-focused principles underlying the curriculum and their ease in discussing complex and sensitive issues around race, bias and systemic barriers (Goode, Ivey, Johnson, Ryoo, & Ong, 2020a;Goode, Johnson, & Sundstrom, 2020b). With additional years of experience, teachers also develop a better understanding of their own school and district in terms of local needs and relevant issues, as well as educational policies and resources available. Providing a strong professional development program for teachers and supporting them as they deliver the curriculum and navigate their specific educational contexts is vital for the success of culturally relevant and responsive computing in the classroom. Future research focused on teacher voice will also be of great importance to ensure that educators receive the full range of support they require to implement the curriculum.

Discussion
We used a Quick Scoping Review (QSR) method to investigate computing curricula that have been developed to be culturally relevant and responsive to a diverse group of learners. Specifically, we aimed to answer the following questions: 1) Which design principles have been embedded into the curricula to promote cultural relevance, equity and justice in computing? 2) How has the success of the teaching approaches or curricula been evaluated? 3) Which factors positively or negatively influence the success of the teaching approaches or curricula? We will begin by considering the first question, before turning to the issues of evaluating success.

Design principles
Across the 12 studies included in the QSR, the most common elements of the design were a) identity development and expression, and b) building on community or cultural knowledge. These elements are central to all culturally responsive teaching, allowing students to see themselves and their communities within computing and recognising and celebrating different types of knowledge and understanding (Madkins et al., 2020). In particular, the interventions focused on Culturally Situated Design Tools (CSDTs) emphasise these elements.
From the earlier set of studies reviewed by Morales-Chicas et al. (2019), the out-of-school programs (Ashcraft et al., 2017;DiSalvo et al., 2014;Scott & White, 2013) tended to focus on more vernacular culture and provided more opportunities for professionally authentic computing experiences, aiming to improve the learners' understanding of different careers and types of roles within computing. They also provided opportunities to identify and try to address issues that were meaningful to the learners and their communities, thus highlighting the relevance of computing and encouraging learners to become agents of change. From these studies, it could perhaps be inferred that non-formal learning spaces are more able to focus on these types of projects, given that they are less constrained by the curriculum and other pressures of teaching within the formal schooling system.
However, the later studies found through the QSR revealed that opportunities to highlight the personal and sociopolitical relevance of computing, to position learners as active agents of change, and to engage in openended and problem-solving activities is indeed possible within a formal computing curriculum (McGee et al., 2018;Ryoo, 2019;Qazi et al., 2020). The ECS curriculum incorporates these elements of culturally responsive teaching throughout its different units. This suggests that balancing the teaching of computing concepts and taking a culturally responsive approach can be done in the classroom but requires intentional and sensitive design.

How are curricula evaluated, and which factors affect their success?
Focusing on our second two questions, their interdependence means it is preferable to explore them together. The main outcomes measured across all 12 studies included in the QSR were related to student attitudes towards computing, with most studies using a survey measure or in-depth qualitative analyses to evaluate attitude change. Eglash et al. (2013) identified that a vernacular approach was relatively more popular with young people than the heritage approach, highlighting the importance of thinking about learners through an intersectional lens that takes into account a range of characteristics, and not focusing solely on heritage or ethnic background. DiSalvo et al. (2014) reported improved confidence in computing after their games testing workshops. One of the key factors in the changing attitudes of the young people was how they viewed participating as games testers as "cool" and that they were "paid to play" (p. 302), allowing them to present their enjoyment and interest in computing to their peers or families with pride rather than embarrassment. Thus, the people around the learners are also vital to the success of more vernacular approaches to teaching computing.
The qualitative studies additionally explored themes of identity, feelings of belonging to computing as a discipline, and changes to learners' understanding of computing concepts over time (Ashcraft et al., 2017;Ryoo, 2019;Yang et al., 2021). Yang et al. identified the importance of collaborative work in increasing feelings of belonging and promoting shared skills and knowledge. Open discussions concerning sociocultural issues surrounding computing were also key to the success of the programs, both through the teachers' willingness and competence to discuss these issues and the students' feelings of comfort and trust to voice their own opinions (Ashcraft et al., 2017;Ryoo et al., 2019). Ashcraft et al. note that it takes time to develop the trusting relationships needed for these sorts of discussions, which suggests that opportunities should be embedded throughout a curriculum and not be presented as a one-off lesson or only early on in a course. This may seem to lend itself more to a non-formal setting which can take place over longer periods of time than students often receive for computing in schools. However, the Scratch Encore and ECS curricula demonstrate that careful planning and consideration can ensure that both computing content and opportunities for cultural responsiveness are incorporated over the course of a school year.
Two studies used objective measures of learning gains, providing evidence that content was covered more quickly using the culturally responsive approach (Davis et al., 2019) and that learners improved their computing understanding and skills over the year of the ECS course (McGee et al., 2018). Both of these studies identified the importance of the teacher in implementing culturally responsive teaching: the quantitative analysis conducted by McGee et al. revealed that years of teaching computing was a key predictor in student learning gains, while Davis et al. discuss the reluctance of the teacher to engage with certain elements of the sociopolitical content of the curriculum and to allow learners to choose their own projects. This reluctance may have been associated with the lack of change in student attitudes towards computing, despite their increased speed of learning. Recent studies have highlighted how important professional development is within the ECS program in terms of fostering a critical approach to educational practices, computing pedagogy, and teachers' own implicit biases (Goode et al., 2020a(Goode et al., , 2020b. The teachers' understanding of culturally responsive teaching, the acceptance of its principles, and their proficiency in implementing them in the classroom are therefore key factors in the success of a culturally responsive approach to teaching computing. However, it is also important to consider the contexts in which teachers are delivering the curriculum, including school and district policies concerning teaching, and the time and resources allocated to computing. Research considering these aspects of teacher constraints will be an important next step in understanding how best to deliver culturally relevant and responsive computing in formal education.

Conclusion and implications for practice
Our review of studies reveals a number of different approaches to incorporating culturally responsive teaching into computing, ranging from short, focused interventions with targeted underrepresented groups, to full curricula implemented in the computing classroom. Most studies used a quantitative approach to evaluate the success of the programs, either through surveys of changing attitudes or objective measures of learning gains. Positive changes in these measures tended to be more evident for longer interventions in which culturally responsive teaching was fully embedded rather than competing with the computing concepts being taught. Nevertheless, further research is required with larger samples and longer-term interventions, in many cases, and using validated instruments to investigate the effect of culturally responsive computing on students' attitudes and their later subject and career choices.
The studies utilising in-depth qualitative analyses revealed more about how the learner experience affected their changing attitudes towards computing: being able to openly discuss difficult subjects, think about the sociopolitical context of computing through a complex, intersectional lens, and being able to collaborate and share knowledge and opinions were central to learners' improving attitudes towards computing. Further qualitative studies are required in the future to provide more of this rich and detailed data concerning how and why attitudes may be changing amongst learners following a culturally responsive computing approach.
Future developments of culturally responsive approaches to teaching computing should incorporate the successful elements outlined above into their curricula. Furthermore, developing teacher knowledge and confidence in culturally responsive approaches will be key to their successful implementation. Professional development opportunities should be created alongside curricula to empower teachers and to allow them to fully support their learners in becoming innovators and agents of change within computing. Further research should also be undertaken to better understand teacher voice and the constraints on, and implications of, delivering culturally responsive computing in the classroom. By taking this approach, it may be possible to improve the diversity of representation amongst students choosing to continue with computing as a subject and a career.