Examining Equity in an Elementary School Computer Science Intervention Using Component-Based Research

This article reports on implementation of a problem-based learning intervention developed with the intention of finding time for computer science (CS) in the elementary school day. This study investigated differences in effects on students in particular socio-demographic groups using a quasi-experimental design. We first provide an overview of the perennial problem of group differences or “gaps” in student outcomes. Then we illustrate how, using componentbased research (CBR), we moved beyond the question of whether the intervention worked, to focus on which parts of the intervention worked, for whom, and under what conditions. Using hierarchical linear modeling, this study draws from a sample of 16 elementary schools with 321 teachers and 5791 students in Broward County, Florida, the sixth largest school system in the United States. This study complements a previous paper (Century, Ferris, Zuo, 2020), which examined associations between intervention components and student outcomes by investigating how outcomes differ for students in different socio-demographic groups and whether the presence of particular intervention components amplify or reduce differences. Through CBR, our work illustrates that CS interventions which may appear to benefit students overall, may be less beneficial or even detrimental to particular groups.


Differences in Academic Outcomes by Group Exist and Persist
Differences in academic outcomes between groups of learners are well-documented across racial, ethnic, and gender identities, and socioeconomic, disability, English proficiency, and migrant status (Celeste et al., 2019;Chmielewski, 2019;Gilmour, et al., 2019;Hung et al., 2020;Owens, 2018). Although limited publications report on group differences in computer science (CS) outcomes at the primary grade levels, these differences are clear in other subjects. In the US, the 2019 National Assessment of Educational Progress (NAEP) showed that White students scored 26 and 28 points higher than their Black peers in 4 th grade and 8 th grade reading assessments, respectively, and 25 and 32 points higher in 4 th and 8 th grade math assessments, respectively (NAEP, 2019). Group differences in achievement outcomes also exist between Hispanic and White students with 21 and 20 point differences for reading in 4 th and 8 th grades, respectively, and 18 and 24 point differences for 4 th and 8 th grade mathematics, respectively (NAEP, 2019). These racial and ethnic group differences can be explained, in part, by differential access to high quality education and inequitable learning experiences. In the US, students who are Black or Hispanic are more likely to attend high-poverty schools (Orfield et al., 2016;Orfield & Lee, 2005) and be denied access to STEM and CS courses and inclusive pedagogy, which often results in segregated CS education and career pathways (Margolis et al., 2017). School poverty rates are also associated with the size of the difference in achievement by racial group and the rate at which these differences increase over time (Reardon et al., 2019).
Given the strong associations between achievement and school-level poverty, it is also important to consider achievement differences across income groups. One study found that for every $1,000 increase in family income, there was an associated 2.1% increase in students' math scores and a 3.6% increase in students' reading scores (Dahl & Lochner, 2012). Additionally, academic outcome differences between students of high-and low-SES groups tend to increase as students get older (Caro, 2009). Unsurprisingly, the 2018 Program for International Student Assessment indicates that students from low SES backgrounds lag far behind more advantaged students in their reading, mathematics, and science performance (OECD, 2020).
Disparities in academic achievement between gender groups also persist. For instance, only 19% of female students scored at a proficient level on the science NAEP test compared to 25% of male students (NAEP, 2019). Similar patterns were observed for scores on the NAEP mathematics test, with only 38% of female students scoring at or above proficiency, compared to 44% of males (NAEP, 2019). Differences also emerged based on English-language proficiency levels. Regardless of grade level, English Language Learners (ELLs) performed lower on reading and mathematics assessments compared to native English speakers, and such differences were particularly pronounced in domains that require more use of the English language, such as reading and writing (Abedi, 2002;Fry, 2007).

Group Differences in Attitudes
In STEM subjects, research on group differences in attitudes have had mixed results, and some group comparisons have yet to be investigated (e.g., ELLs and native English speakers). For instance, some studies indicate that Black students report lower science ability and mathematics beliefs compared to White students (Britner & Pajares, 2001;Pajares & Kranzler, 1995), whereas others have found that, compared with White students, Black students demonstrate higher mathematics attitudes (e.g., mathematics values), and Hispanic students exhibit higher mathematics interest and intrinsic motivation (Else-Quest et al., 2013;Stevens, Olivárez Jr, & Hamman, 2006). Another study found that Black and Hispanic boys specifically, reported lower levels of educational aspirations, persistence, and self-efficacy related to science compared to boys from more economically advantaged backgrounds, White boys, and girls from all backgrounds (Perry et al., 2012). Still other research indicates both Black and Hispanic students have a similar level of interest in STEM majors as their White counterparts (Anderson & Kim, 2006;Hurtado, et al., 2010). Moreover, female students generally report less motivation and confidence in their CS and mathematics-related skills and abilities to learn these subjects compared to male students (Beyer, 2014;Cunningham et al., 2015;Doubé & Lang, 2012;Hur et al., 2017;Maste et al., 2016). Girls and women are also more likely to show math and spatial anxiety (Ferguson et al., 2015;Maloney et al., 2012;Sokolowski et al., 2019). Together, this collection of results underscores the importance of pursuing more research on discrepancies in STEM-related attitudes between different groups of students.

The Importance of Addressing Group Differences in Computer Science
The need to increase the number of learners interested in CS has never been greater (Scott et al., 2017;Vakil, 2018). Computing knowledge is essential to address the growing demand for CS skills in STEM jobs, yet we must also acknowledge that women and underrepresented minorities earn far few degrees in computing fields at this writing (Sax et al., 2018). Moreover, discrepancies in access to technology in the home and CS educational experiences persist, and are particularly pronounced for Black and Hispanic students and youth from lower SES families (Wang et al., 2016). Together, these findings suggests that research attention must be paid to diverse learners' experiences with STEM and CS, teachers' and administrators' roles in broadening participation in CS, instructional practices facilitating inclusive practice and CS learning, and educational structures and policies contributing to equitable CS educational opportunities (Ryoo et al., 2019;Santo et al., 2019).
It is also essential to begin CS education in the early grades. Introducing young learners to even basic CS concepts (i.e., plugged and offline, unplugged lessons) is crucial during the elementary school years for several reasons. First, doing so helps form the foundation for future learning in the subject, and second, it allows students to experiment with a complex and challenging subject before they form opinions of their abilities (Prottsman, 2014). There is little research on CS education in the early grades and even less that examines which specific learning experiences contribute to diverse students' pursuit of CS. The current study addresses these gaps in the literature by exploring the CS experiences of diverse learners in the early grades. This work will further increase our understanding of how to promote educational equity in CS education as measured by academic and attitudinal outcomes.

Approaches to Examining Group Differences in Outcomes
Prior research considering group differences in academic achievement and attitudes has employed a variety of research approaches. However, many studies simply describe differences in outcomes and studies that focus on comparing outcomes, often privilege certain socio-demographic groups, namely White, male students, over others (Gutiérrez & Dixon-Román, 2011). Such 'gap gazing' illuminates the presence of differences, but does not provide any insights as to the nature of disparities that can inform efforts to change them (Lubienski, 2008).
Research that goes beyond 'gap gazing' operates on the assumption that instructional practices, programs, and educational interventions will not impact all students in the same way because members of diverse populations have unique needs (Gutiérrez, & Jaramillo, 2006;Rodríguez et al., 2016). Research methods need to account for the nuances that happen when programs are implemented (Vossoughi et al., 2016). To do so, researchers must utilize diverse methodological approaches that move beyond a sole focus on "what works" (Bullock, 2012) to also consider what works for whom and under what conditions. Approaching the study of CS achievement and attitudinal outcomes in this manner will elevate the importance of achieving equitable outcomes across diverse groups of learners.

Material Studied
This research was carried out through a research-practice partnership (RPP) focused on finding time for CS in the elementary school day, which is often dominated by literacy and mathematics instruction. The strategy was to create interdisciplinary problem-based learning modules that would be taught during the literacy block. The modules were organized around a science or social studies theme and had science, English/Language Arts, and associated CS lessons. The CS lessons were part of the Code.org courses available worldwide.
We referred to the modules as "Time4CS" Modules (see Century, Ferris, & Zuo, 2020 for a more detailed discussion of the modules). During the study, third-fifth grade teachers used two Time4CS Modules during the 2016-2017 school year. Each module was taught over a 5-8 week period and was composed of five lesson "collections." Each collection had four 60-minute lessons written in the 5E Model (i.e., Engage, Explore, Explain, Extend, Evaluate; Bybee, 2009) and had at least one CS lesson. Each module ended with a culminating project that entailed using CS to create a final product highlighting what students learned.
Following a quasi-experimental design, the project's primary research questions explored outcome differences for students in classrooms where teachers implemented and did not implement the Time4CS Modules. These analyses found that teachers who used the Time4CS Modules taught more CS than teachers in the comparison group. However, no differences in attitudinal or academic achievement outcomes were observed between students in treatment and comparison groups (Century, Ferris, & Zuo, 2020). Because this was the case, we utilized component-based research (CBR) to conduct further, in-depth analysis to explore associations between specific components of the Time4CS Modules and student outcomes. Using CBR revealed positive associations between particular Time4CS Module components and students' attitudinal and achievement outcomes. For example, we found positive associations between teachers' use of interdisciplinary teaching practices and students' achievement outcomes (Century, Ferris, & Zuo, 2020). In the findings reported here, we used CBR to conduct follow-up analyses that move beyond the aforementioned findings to deeply examine differences in student outcomes based on socio-demographic characteristics and the effect of teachers' instructional practices on outcomes for students in different groups. To do so, we investigated two additional research questions: 1. Are there differences in students' attitudinal and academic achievement outcomes based on sociodemographic characteristics?
2. Does the implementation of particular Time4CS Module components amplify or reduce group differences in students' attitudinal and academic achievement outcomes?

Area and Sample Description
The

Theoretical Approach -Component-Based Research
This study used a component-based research (CBR) approach. CBR deconstructs an innovation into its components, organizes those components into categories, and uses the categories as a foundation for looking beyond whether the innovation "worked," to examine what parts of an innovation work, for whom, and under what conditions. In this study, CBR was guided by an innovation implementation framework (Century, Cassata, Rudnick, & Freeman, 2012) that organized the components of the Time4CS Modules into two main categories -structural (e.g., number of CS lessons taught, lesson omission) and interactional (e.g., teacher facilitation of cognitively-demanding work, teacher facilitation of student intellectual risk taking). See details in Table 3. We also measured contextual factors hypothesized to influence teachers' Time4CS Module implementation (e.g., teachers' years of teaching CS, perception of their own innovativeness or resourcefulness) utilizing a framework that outlines factors that influence implementation (Century & Cassata, 2014). By using CBR, we were able to identify the impact of specific innovation components on students' attitudinal and achievement outcomes and disentangle associations between specific components and outcomes for particular student groups. Employing CBR enabled us to move beyond "gap gazing" to generate greater understanding about potential ways to reduce group differences in attitudinal and achievement outcomes across diverse groups of students.

Teacher facilitation of group work
How often did you encourage group members to solve problems together?
Teacher facilitation of cognitively demanding work How often did you ask students to explain how they solved a problem?
Teacher facilitation of students taking intellectual risks How often did you ask students to answer a question even if they were unsure?
Teacher use of interdisciplinary teaching practices How often did you explicitly cover standards from multiple subject areas in the same lessons?
Teacher facilitation of group work How often did you encourage group members to solve problems together?

Measures and Data Sources
Teacher Implementation Questionnaire: This questionnaire, administered in spring 2017 using Qualtrics ©, assessed Time4CS Module structural and interactional component implementation and factors influencing implementation, such as teacher attitudes and demographic characteristics. All items and response scales included in the Time4CS Teacher Questionnaire are listed in Appendix A.
Student Attitude Questionnaire: Students completed questionnaires at the beginning and end of the 2016-2017 academic year using Qualtrics ©. These questionnaires assessed students' attitudes about school (i.e., school affinity, school self-efficacy) and CS (i.e., CS affinity, CS self-efficacy, CS identity, and CS utility) using items scored on a five-point Likert scale. These questionnaires also asked about previous experience with Code.org CS activities (i.e. yes/no) prior to the study. All items and response scales included in the Time4CS Student Questionnaire are listed in Appendix B.
Florida Standards Assessment (FSA) and Achieve3000 scores: Measures of student academic achievement included the Florida Standards Assessment (FSA) and a diagnostic tool BCPS used called Achieve3000. The state administers FSA assessments in ELA and mathematics in 3 rd , 4 th and 5 th grades. The FSA science test is administered in 5 th grade, only. The FSA assesses student knowledge and critical thinking skills. The Achieve3000 literacy test is administered three times a year to assess students' reading level as a means to support differentiated instruction. In both cases, BCPS de-identified and shared these data for the 2016-2017 academic year with the research team.
Administrative Data: BCPS also shared students' de-identified socio-demographic data, including gender, ethnicity (i.e., non-Hispanic and Hispanic status), race, ELL proficiency, and socio-economic status (i.e., free and reduced-price lunch [FRL]). Students also self-reported their gender, ethnicity, and race, and we cross-referenced these data with the district data to account for missing values.

Using CBR to Measure Time4CS Module Implementation
Studies of CS and other school interventions do not typically measure intervention implementation and when they do, implementation measurement is only conducted with treatment groups. However, CBR recognizes that some intervention components may in fact be accessible to teachers in the comparison group, and therefore, must be measured as well. This was the case in this study. Specifically, the Code.org CS lessons associated with the Time4CS Modules were publicly available to all teachers in the district. Similarly, the Time4CS Modules' interactional components included instructional practices (e.g., facilitating group work, using interdisciplinary teaching practices) that any teacher can use, with or without the Time4CS Modules. Therefore, while teachers in the treatment condition were asked to report on implementation of all module components, teachers in the comparison condition were asked to report on the components (and related factors) they could have enacted even though they did not implement the modules.
Structural implementation components: Teachers in the treatment group reported whether they completed (i.e., yes/no) the Time4CS Module with their students. They also reported on how many module lessons they completed and the sections of those lessons they used or omitted. Moreover, because we used a CBR approach, teachers in both treatment and comparison groups reported the number of grade-level assigned and non-grade-level assigned Code.org CS lessons they completed as well as whether they completed (i.e., yes/no) any additional, non-Code.org CS activities. The number of grade-level assigned, and non-grade-level assigned Code.org CS lessons were converted into completion percentage scores by dividing the number of lessons teachers reported completing by the total number of lessons available to complete. Higher completion percentage scores represented greater CS exposure for students.
Interactional implementation components: All teachers reported on their use of interactional Time4CS Module components. These components entailed the instructional strategies embedded in the module, but as explained above, could also be carried out in "business-as-usual" (comparison) classroom instruction. The questionnaires asked teachers to report on use of interdisciplinary teaching practices (e.g., How often did you explicitly cover standards from multiple subject areas in the same lesson?) as well as their facilitation of group work (e.g., How often do you encourage group members to work to solve problems together?), cognitively-demanding work (How often did you ask students to explain how they solved a problem?), and intellectual risk-taking (e.g., How often did you ask students to answer a question even if they were unsure?). Each scale used a five-point Likert scale (1 = Never, 5 = Always) with higher scores representing greater use of each component.
Influential Factors: The factor framework mentioned in section 4.1 organizes influential factors (i.e., contexts and conditions) into several levels (i.e., individual, organizational, and environmental) (Century & Cassata, 2014). The questionnaires asked teachers to report on three individual (teacher) factors, including innovativeness (e.g., I experiment with new practices all the time), resourcefulness and coping (e.g., I am able to manage the pressure and stress at my school well), and years of teaching CS. Each scale used a six-point Likert scale (1 = Strongly disagree, 6 = Strongly agree) with higher scores representing greater use of these practices.

Analytic Strategy
To conduct our analysis, we used Hierarchical Linear Modeling (HLM) with HLM7 software (Raudenbush, Bryk, Cheong, Congdon, & Toit, 2011). First, two-level, intercept-as-outcome models were conducted, which identified significant group differences in achievement and attitudinal outcomes based on students' socio-demographic characteristics (i.e., Level-1 variables) 1 . Next, two-level, random-intercept and coefficient-as-outcome models were conducted to examine the interactive effects between teacher-level implementation of particular interactional Time4CS Module components (i.e., specific instructional practice variables) and students' socio-demographic characteristics (Raudenbush & Bryk, 2002). In this second set of analyses, the Level-1 intercepts were no longer modeled as a function of structural or interactional component implementation or factors at Level-2 as in the first set of analyses; instead, they were included as random effects. More specifically, significant Level-1 coefficients for students' socio-demographic characteristics identified in the first set of analyses were modeled as a function of structural or interactional component or factor variables at Level-2 in the second set of analyses.
In all analytic models, students' socio-demographic characteristic variables were entered at Level-1. These variables were dummy coded as follows: gender (reference group: male); race (reference group: White students); ethnicity (reference group: non-Hispanic students); SES (reference group: students who do not receive free and reduced-price lunch); and ELL proficiency (reference group: native English speakers). Students' previous experience completing Code.org activities was included at Level-1 as a control variable as were their socio-demographic characteristics.
Students' attitudes about school and CS collected during pre-questionnaire administration (i.e., Fall 2016) were also included at Level 1 in models where attitudinal outcomes were examined. Level-2 teacher variables included structural and interactional implementation components and factors. All categorical variables were uncentered, whereas continuous variables were centered around the group mean for Level-1 variables and the grand mean for Level-2 variables.

Results
Significant findings are reported in text. Standardize beta coefficients and standard error terms are reported for all associations between students' socio-demographic characteristics (i.e., Level-1 variables) and their academic achievement and attitudinal outcomes are included in Table 4 and 5, respectively. In Table 6 and 7, we report all standardized beta coefficients and standard error terms for the interactive effects between students' socio-demographic characteristics (i.e., Level-l variables) and structural and interactional components of Time4CS module implementation and factors (i.e., Level-2 variables) 2 . The significant group differences reported in-text and in Tables  4-7 emerged when controlling for all other student socio-demographic characteristic variables in the analytic model.
Race: On the mathematics FSA, Asian students (β = 5.92, p < 0.01) scored significantly higher, and Black students (β = -8.42, p < 0.01) scored significantly lower than White students. On the ELA FSA, Black students scored significantly lower than White students. This main effect was qualified by a significant interaction, which indicated that this score differential (i.e., gap) on the ELA FSA between Black and White students was larger in classrooms where teachers reported completing a higher percentage of grade-level assigned CS lessons (a structural component) (β = -34.69, p < 0.01).  On the science FSA, Asian students scored significantly higher compared to White students. This main effect was qualified by a significant interaction indicating that this score differential on the science FSA between Asian and White students was larger in classrooms where teachers reported greater use of interdisciplinary teaching practices (interactional component) (β = 55.60, p < 0.05). On the other hand, Black students scored significantly lower on the science FSA compared to White students. This main effect was qualified by a significant interaction, which indicated that this score differential on the science FSA between Black and White students was smaller in classrooms where teachers reported higher levels of resourcefulness and coping (influential factor) (β = 10.45, p < 0.05).
SES: Students receiving FRL scored significantly lower on Achieve3000 Lexile scores (β = -31.03, p < 0.05) and the science FSA (β = -6.58, p < 0.01). Students receiving FRL scored significantly lower on the ELA FSA as well, and this main effect was qualified by a significant interaction, which indicated that this score differential on ELA FSA between students receiving FRL and those who do not was larger when teachers completed more additional (non-Code.org) CS activities (structural component) (β = -6.41, p < 0.01) and reported higher levels of resourcefulness and coping (influential factor) (β = -5.23, p < 0.01). In addition, students receiving FRL scored significantly lower on the mathematics FSA. This main effect was qualified by a significant interaction, which indicated that this score differential on mathematics FSA between students receiving FL and those who do not was smaller when teachers reported higher levels of facilitation of group work (interactional component) (β = 4.84, p < 0.05).
ELL Proficiency: ELLs (β = -8.78, p < 0.01) scored significantly lower on the science FSA than native English speakers. ELLs scored significantly lower on the ELA FSA than native English speakers. This main effect was qualified by a significant interaction, which indicated that this score differential on ELA FSA between ELLs and native English speakers was larger when teachers reported higher levels of facilitation of group work (interactional component) (β = -9.75, p < 0.01). However, this score differential on ELA FSA was smaller when teachers reported higher levels of resourcefulness and coping (influential factor) (β = 5.92, p < 0.05). Furthermore, ELLs scored significantly lower on the mathematics FSA. This main effect was qualified by a significant interaction, which indicated that this score differential on mathematics FSA between ELLs and native English speakers was smaller when teachers reported completing a higher percentage of non-grade-level Code.org CS lessons (β = 30.13, p < 0.01) (structural component), facilitating higher levels of student intellectual risk taking (interactional component) (β = 9.98, p < 0.01), higher levels of resourcefulness and coping (influential factor) (β = 6.60, p < 0.05), and more years of teaching CS (influential factor) (β = 1.77, p < 0.05). On the other hand, this score differential on mathematics FSA was larger when teachers reported higher levels of facilitation of group work (interactional component) (β = -12.48, p < 0.01).

Attitudinal Outcomes
Gender: No significant differences were observed for students' attitudinal outcomes.
SES: Students receiving FRL reported more positive CS affinity attitudes. This main effect was qualified by significant interactions, which indicated that this difference in CS affinity attitudes between students receiving FRL and those that do not was larger in classrooms where teachers taught the Time4CS Modules (whole intervention) (β = 0.47, p < 0.05), and when teachers reported greater use of interdisciplinary teaching practices (interactional components) (β = 0.46, p < 0.01) and more years of teaching CS (influential factor) (β = 0.09, p < 0.01). In addition, students receiving FRL reported more positive CS identity attitudes (β = 0.16, p < 0.05).
11 Note. * p < .05. Standardized beta coefficients and standard error (SE) terms are presented in the table with SE reported in parentheses. We controlled for students' ethnicity (i.e., Hispanic and non-Hispanic identity) in all models. No significant group differences emerged for ethnicity; therefore, it was not included in any interaction terms at Level-2. ELL = English-language learner. 12 Table 7. Results from the two-level, random-intercept and coefficient-as-outcome model examining students' CS affinity attitudes Group differences were only observed for students' socio-economic status. We controlled for students' ethnicity (i.e., Hispanic and non-Hispanic identity) in all models. No significant group differences emerged for ethnicity; therefore, it was not included in any interaction terms at Level-2. ELL = English-language learner.

Academic Achievement Gaps Persist
Considerable efforts have been made to address inequities in achievement outcomes for diverse populations of students; however, gaps persist. Unsurprisingly, and consistent with previous research, findings from the current study indicate that Black students, students from economically disadvantaged backgrounds (i.e., students receiving FRL), students acquiring English as a second language, and female students scored lower on some achievement outcomes compared to their reference groups (Hadden et al., 2020;Xu, 2019). This study addresses calls from prior research (e.g., Lubienski, 2008) by moving beyond identifying group differences in outcomes to offering potential reasons why such gaps persist. Employing CBR in this study enabled us to identify the ways different structural and interactional components of the Time4CS Modules influenced the degree of group difference (i.e., amplifying versus reducing it).

Structural Components of the Time4CS Modules Contributed to Larger or Smaller Group Differences in Outcomes
This study found that CS-related components (e.g., implementation of grade-level assigned Code.org lessons; implementation of non-grade level assigned Code.org lessons; implementation of additional non-Code.org lessons) influenced whether the group difference was amplified or reduced. Our earlier analysis (Century, Ferris, & Zuo, 2020) showed no associations between the percentage of grade-level assigned Code.org lessons and FSA outcomes. However, by disentangling the effect of student group, we observed group differences between Black and White students for mathematics FSA outcomes, and this difference was amplified in classrooms where teachers reported teaching more grade-level assigned CS lessons. In BCPS, similar to other large, urban public-school districts, it is possible that disproportionate numbers of Black students reside in low-income areas and attend under-resourced schools with fewer educational supports. When teachers dedicated more time within the literacy block to grade-level assigned CS lessons, this in turn, may have reduced the time Black students had available for instruction in FSA tested areas (e.g., mathematics). This finding suggests Black students face compounded inequities; attending underresourced schools limits the instructional support present at the outset so when opportunities to engage in CS are presented, outcomes in these other subject areas may suffer.
We also observed through use of CBR that teaching "additional CS activities" was associated with lower ELA FSA scores for students receiving FRL. Because this group by definition come from families with lower incomes and/or parental educational attainment, they also reside in, and attend, more under-resourced neighborhoods and schools. It is possible that the presence of additional CS activities took time away from the particular kinds of ELA instructional supports these learners needed to score highly on the state-administered examination.
Moreover, our results about non-grade level assigned Code.org CS lessons underscore additional equity-related complexities. Our earlier analysis (Century, Ferris, & Zuo, 2020) illustrated that non-grade level assigned Code.org CS lessons were positively associated with students' ELA and mathematics FSA scores. However, when examining specific student groups, the presence of more non-grade level assigned Code.org CS lessons reduced the group difference in mathematics FSA between ELLs and their native English speakers. This group difference may have been reduced for several reasons. For instance, teachers may have selected non-grade-level assigned Code.org CS lessons that complemented and enhanced the curricula they were already teaching, in turn allowing them to provide more appropriate scaffolding for language learners. Or, they may have selected non-grade-level Code.org CS lessons to support their own, and their students', varied experiences, and interests. Emphasizing students' diverse experiences contributes to the creation of an inclusive classroom environment where teaching reflects the diversity of needs of all students and teachers consider how to include rather than exclude learners who experience difficulty (i.e., inclusive practice; Abels, 2019), which is central to cultivating positive learning experiences with ELLs (de Jon et al., 2013). Teachers may also have felt that their students were capable of completing these optional lessons, which at times covered more-advanced CS content, because they felt that their students had the ability to understand morechallenging CS concepts. Thus, more non-grade level Code.org CS lessons may have been conducive to their learning, engagement, and growth as students, overall, as well as reducing potential outcome differences based on language proficiency. Conversely, teachers may have carried out more of these lessons covering less-advanced content (i.e., below grade level: fourth grade students completing non-grade level assigned lessons intended for third graders) because they believed they were more-closely aligned with their students' abilities. Additional studies focusing specifically on academic achievement and STEM and CS attitudes are needed with ELLs in the future to better understand how to create equitable learning opportunities for this growing number of students and foster teachers' inclusive practice in their classrooms.
Significant attitudinal effects only emerged in the presence of the structural component related to whether teachers taught the Time4CS Module. Specifically, FRL students demonstrated higher CS affinity attitudes, and this difference was larger in classes where the Time4CS Module was taught. This finding suggests that although we found no differences between treatment and comparison groups as a whole (Century, Ferris, & Zuo, 2020), it appears that for students receiving FRL, engaging with the Time4CS Module (e.g., exposure to CS topics presented in an interesting manner) was beneficial for their CS affinity.

Interactional Components of the Time4CS Modules Contributed to Larger and Smaller Group Differences in Outcomes
We also observed interaction effects with some of the interactional Time4CS Module components. For example, for mathematics FSA outcomes, teachers' facilitation of student intellectual risk-taking contributed to reducing group differences in outcomes between ELL students and native-English speakers. These findings suggest that ELLs may benefit from opportunities to take risks during instruction, and that those opportunities may positively contribute to better academic achievement. Accordingly, teachers who create intellectually safe and supportive learning environments when teaching CS may positively shape students' overall outlook on learning and motivation to persevere when faced with uncomfortable learning challenges (Rabe, 2018 Findings related to teachers' facilitation of group work were mixed. For example, when teachers reported more facilitation of group work, the group difference in mathematics FSA scores between low and high SES students was reduced. This finding supports prior research suggesting that group work promotes academic achievement for disadvantaged students (Bailey & Bradbury-Bailey, 2006). Through group work, these students may receive extra peer-to-peer support when working with their classmates as well as increased opportunities to take ownership of work aligning to their skillsets (Slavin, 2014). However, when teachers reported more facilitation of group work, the difference widened between ELLs and Native-English speakers. This effect also emerged when examining ELA FSA scores. These findings may have resulted because group work requires greater communication and collaboration, which are areas where ELL students may be at a disadvantage due to their more limited English-language proficiency.
In addition, findings on interdisciplinary teaching practices further disentangled effects observed for all students in our previous paper (Century, Ferris, & Zuo, 2020). Our earlier analysis indicated that the presence of interdisciplinary teaching practices was positively associated with ELA FSA outcomes for the whole student population (Century, Ferris, & Zuo, 2020), whereas the current analysis revealed that the presence of these practices contributed to two instances of amplified group differences. Specifically, on the science FSA, the group difference between Asian and White students was larger in classrooms where teachers' reported using more interdisciplinary teaching practices. Some research (Farris, 2015) has suggested a link between interdisciplinary teaching practices, multiculturalism, and culturally-responsive instructional strategies, which may be beneficial for Asian students' academic achievement, in particular (Change & Le, 2010). Additionally, the group difference in CS attitudes between FRL students and their counterparts was larger when more interdisciplinary teaching practices were present. The reasons behind these findings are unclear, but we recommend that future researchers use CBR to investigate these effects more specifically to gain a better understanding of the role of interdisciplinary instructional practices.

Influential Factors Also Contribute to Larger or Smaller Group Differences in Outcomes
Two teacher characteristics contributed to changes in group differences in outcomes. First, the group difference in science FSA scores was reduced between Black and White students in classrooms where teachers reported having more resourcefulness and coping skills. Similarly, when teachers reported more resourcefulness and coping skills, the group difference between ELL and native-English speakers on mathematics FSA was reduced. On the other hand, the group difference on the ELA FSA between FRL students and their counterparts was larger in the presence of more teacher resourcefulness and coping. One can speculate that teachers who perceive themselves as being resourceful and having coping abilities might be more adept at inclusive practice (Bentley-Williams, Grima-Farrell, & Laws, 2017), in turn providing Black and ELL students with the opportunities and resources they need to increase their academic achievement. Seeing the findings for FRL students; however, suggests that these benefits are not universal, for reasons this study did not reveal and that should be the focus of future investigations.
More commonly measured teacher characteristicsyears of teaching CSwere also associated with reductions in group differences. We observed smaller group differences between ELL students and their native-English speaking counterparts for mathematics FSA and ELA FSA scores. While neither of these characteristics was revealed to have direct effects on student outcomes (Century, Ferris, & Zuo, 2020), one can speculate that for ELL students in particular, having teachers with more experience with CS content is beneficial.

Conclusion
The current study explored differences in students' attitudinal and academic achievement outcomes based on teachers' implementation of the Time4CS Modules in their classrooms. We employed CBR to disentangle associations between specific components of the Time4CS Modules and students' attitudes and academic achievement. Our results assess beyond whether the Time4CS Modules worked and indicate that academic achievement and CS attitudinal gaps persist. Utilizing CBR allowed us to identify potential reasons why some components of the Time4CS Modules (i.e., intervention) were beneficial for certain groups of students, but not others. Use of CBR also underscored areas of continued improvement for researchers and educators striving to provide equitable CS learning experiences for young learners. By pinpointing how students' socio-demographic characteristics interact with aspects of their learning context (e.g., teachers' instructional practices), researchers and practitioners can better understand the contexts and conditions needed to promote desired student outcomes related to CS. Therefore, CBR is a valuable approach for moving beyond "gap gazing" to gain insights about what parts of an innovation worked, for which students, and under what conditions. Despite these promising findings and their implications for researchers and educational professionals dedicated to creating more-equitable CS learning opportunities, this project represents merely a starting point. Future research must address the current study's limitations to generate insights that can inform the continued development of CS educational experiences for today's diverse learners.

Limitations and Future Directions
Our findings must be considered in light of several limitations. First, the team was unable to control for the BCPS academic calendar (e.g., standardized testing schedule), which likely impacted implementation of the Time4CS Modules given that some teachers taught the modules before FSAs were administered, whereas others used the module after testing took place. Greater attention must be placed on when teachers are implementing the intervention in future studies so accurate effects of the intervention on outcomes can be generated. Second, the teacher and student questionnaire data is subject to self-report bias and socially-desirable reporting, which may contribute to inflated (e.g., more positive) response patterns in the data ( Van de Mortel, 2008). Future research should adopt multi-method data collection strategies (Bulsara, 2015), including interviews or focus groups with teachers and students. Classroom observations could also be used to capture real-time accounts of teachers' use of different components to better understand the reasons particular components may be associated with larger or smaller group differences (Archer et al., 2016). Third, the Time4CS Modules used were in the early stage of development during the study, and they may have required revisions to improve usability and to ease teachers' learning curve when implementing them.
We also want to acknowledge two considerations for future examinations focused on equity. First, we did not use methods that accounted for intersectionality. Although our results underscore the importance of CBR when disentangling the effect of structural and relational components of a curriculum designed to promote more equitable student outcomes, follow-up investigations are also needed to better understand how different student characteristics intersect (e.g., race, gender). An intersectional conceptual and analytical approach (Else-Quest & Hyde, 2016a; Else-Quest & Hyde, 2016b) was not the primary focus of this investigation. We recognize that the use of additive analytic approaches (Cole, 2009) limits the conclusions that can be drawn regarding students' dual and overlapping identities (e.g., being Asian and female; being Black and male; being multiethnic; being ELL and a FRL recipient). Nonetheless, our findings emphasize the need to look more deeply at group differences and recognize that instructional practices may be beneficial to some groups, but detrimental to others. As a result, we encourage future researchers to employ intersectional conceptual and analytical approaches to simultaneously examine student characteristics, attitudes, and academic outcomes as such investigations have the potential to better inform the creation of strategies that can promote educational equity.
Second, we want to acknowledge the implicit and explicit biases associated with our academic outcome measures, standardized tests (Knoester & Au, 2017). For instance, some studies indicate that standardized tests are inequitable by design (Au, 2010;Morgan, 2016;Well et al., 2019). Thus, focusing on standardized testing outcomes when striving for educational equity across diverse student populations overlooks how the test design itself contributes to group differences. Students' standardized test scores may be misleading and not accurately depict student learning (Morgan, 2016). Future research needs to acknowledge these shortcomings when referencing empirical literature, use more equitable assessments, and investigate a wide variety of educational outcomes, including students' attitudes toward different subject areas, motivations in the classroom, and interests.

The Call for Research that Will Contribute to More Accurate and Useful Findings about Equitable CS Learning Opportunities
In light of these limitations, the findings of this study communicate a powerful message: CBR enables researchers to examine the impact of specific aspects of an intervention for different groups of learners. When studies of CS interventions show positive or negative main effects, it is essential to look deeper through such approaches. Examining the components of educational interventions and their effect on particular student groups will enable us to learn more about how to develop more equitable CS learning opportunities and teachers' inclusive practice. Although CS interventions vary widely, some of them share common components with others. Using a CBR approach can enable the field to accumulate knowledge about these shared components. This will pave the way for researchers and practitioners to develop robust knowledge about the best structures and interactions to use in CS instruction to meet the learning needs of all students.

Module Completion
As part of the STEM+C Integrated Literacy Block project, you were provided with activities and lessons to use during a daily 180-minute integrated literacy block. There were two components of this180-minute block: 1) The 90-minute Reading Block using your normal ELA curriculum and/or suggested texts and center activities provided as part of the STEM+C materials; and 2) The 90-minute interdisciplinary STEM+C Module, which includes the 5E lessons that integrate science, social studies, and computer science.
Throughout the questionnaire, we will use "Integrated Literacy Block" to refer to the entire 180-minutes. We will use "Reading Block" and "STEM+C Module" to refer to the two 90-minute components of the 180-minute integrated Literacy Block. Now we would like you to just think about the STEM+C Module component of the 180-minute Integrated Literacy Block. These are the 5E lesson activities on science, social studies, and computer science.
5. The STEM+C Module lessons include "core" learning resources that you were expected to use as well as "non-core" or supplemental elements that were optional. Please consider the entire STEM+C Module (both the core AND supplemental activities of the 5E lessons) when answering the following questions, unless otherwise stated.
Approximately how many weeks did you spend doing the entire STEM+C Module? (1) ... 12 weeks Each STEM+C Module lesson is intended to take 90 minutes, but we know there will be variations in how much time each teacher actually spends.
6. Do you think 90 minutes was sufficient to complete the STEM+C Module 5E lesson activities as outlined in the Module? 1) Yes 2) No 7. About how much more time (than the 90 minutes) each day do you feel you needed to complete all of the STEM+C Module 5E lesson activities? 1) Less than 15 minutes 2) 15-30 minutes 3) 31-45 minutes 4) 46-60 minutes 5) More than 60 minutes We also know that some teachers worked with media specialists and other "specials" teachers (e.g., art, music) to implement the STEM+C Module 5E lesson activities. For example, some teachers had their media specialist do the online computer science activities that were part of the STEM+C Module.
8. Did you have a media specialist or other specials teacher do any part of the STEM+C Module 5E lessons with your students? 1) Yes 2) No You said you had a media specialist or other specials teacher do parts of the STEM+C Module lessons with your students. Please take those activities into account and answer as best as you can for yourself and on behalf of the specials teacher(s) when answering the following questions. Number of lessons () 11. In all the lessons you taught, did you ever omit one or more of the 5E components (e.g., "Engage," "Explain") for any reason? Do not count 5E components that a specials teacher taught on your behalf as an omission. 1) Yes 2) No 12. About how often did you omit each of the following 5E components for any reason?
(Scale: Never, Rarely, Sometimes, Often, Always) 1) Engage 2) Explore 3) Explain 4) Elaborate/Extend 5) Evaluate 13. You said you omitted some portion of the STEM+C Module. From the list below, please select the top three reasons you did so.
1) The activities were not identified as core parts of the lesson.
2) I didn't have access to the necessary technology (e.g., computers, Internet) to complete the activities. 3) I had access to the necessary technology (e.g., computers, Internet) but it wasn't working. 4) The lesson parts were too long. 5) The lesson parts required too much preparation. 6) The lesson parts weren't written clearly enough to teach them. 7) I didn't understand the content of the lesson parts well enough to teach them. 8) The lesson parts weren't relevant to my students. 9) The lesson parts weren't as important to teach compared to others I needed to teach. 10) Other: (please explain) _______________________________________________

Lesson Modification
Sometimes teachers modify lessons by departing from what is written in the curriculum materials. This includes using different materials to complete the lesson and/or teaching the lessons in a different way than written (e.g., doing the lesson as a whole group activity instead of partner work). Modification does not include omitting or skipping portions of the STEM+C Module.
14. Of the STEM+C Module lessons that you taught; did you modify them in any way? . From the options below, please select the one that best represents how your students used journals. 1) My students used a separate journal for each subject (e.g., one journal for math, one for science, one for literacy, etc.). 2) My students used one journal for all subjects.

ACHIEVE3000
As part of the STEM+C Integrated Literacy Block project, you were asked to administer Achieve3000 diagnostic assessments to students. We'd like to know about your students' use of other (non-assessment) Achieve3000 materials and activities (e.g., nonfiction texts from Achieve3000).
24. On average, since the beginning of the school year, how many times per week did you have your students use Achieve 3000 (not-assessment) materials and activities? 0 If your students worked with a media specialist or specials teacher, please answer on his/her behalf to the best of your ability.

5)
Consider the merits of using different approaches or problem-solving strategies 6) Solve problems in more than one way 7) Communicate their thought process to others 37. During STEM+C Module lessons, about how often did you explicitly ask students to do the following? (Scale: Never, Rarely, Sometimes, Often, Always)

1)
Answer a question even if they were unsure 2) Try new things even if they might make mistakes 3) hare their ideas even if they were different from others Support -Implementation 38. Apart from the professional development workshops at the beginning of the school year, did you receive additional support in implementing the STEM+C integrated module? Please select all that apply. 1) Broward STEM+C Module contacts (i.e., Annmargareth Marousky, Kelly Thomas, and/or Lisa Milenkovic) 2) Your school's STEM+C Module designee 3) Media specialist(s) 4) Other teachers at your school implementing STEM+C Module materials 5) Other teachers from different schools implementing STEM+C Module materials 6) Other teachers not implementing STEM+C Module materials 7) PLC facilitator(s) 8) In-service facilitator(s) 9) School leaders (e.g., principal, assistant principal) 10) Canvas website 11) Online discussion boards or forums 12) Other online resources 13) Other (please explain)________________________________________________ 14) I did not receive any additional support.

Attitudes Toward Interdisciplinary Teaching Practices (Treatment)
Now we're interested in learning about your attitudes toward interdisciplinary teaching practices. The STEM+C Module is an example of interdisciplinary teaching that integrates literacy, science, computer science, and social studies.

Leadership Attitudes Toward CS (Treatment)
We're interested in your perception of your school leaders' attitudes towards computer science. Your answers are confidential and will not be shared with anyone outside of the research team.
44. How much do you agree or disagree with the following statements? My school leaders… (Scale: Strongly Disagree, Disagree, Somewhat Disagree, Somewhat Agree, Agree, Strongly Agree) 1) Think it's important for all students to learn computer science in elementary school.
2) Believe that learning computer science in elementary school will help students get a good job someday.
3) Communicate the value of computer science to students, parents, teachers, and staff. Designing websites, apps, and video games.

3)
Studying how computers work.

4)
Fixing computers when they break.

5)
Building computers and other hardware.

6)
Making spreadsheets and graphs. 7) Using computers to solve complex problems.

11)
Other (please explain): Thanks for telling us what you think computer science is. For this survey, when we say computer science, we mean: Studying and using computers to help solve problems and to create technology, like video games and apps.
51. How much do you agree or disagree with the following statements about your teaching of computer science? (Scale: Strongly Disagree, Disagree, Somewhat Disagree, Somewhat Agree, Agree, Strongly Agree) 1) I understand computer science concepts well enough to be a very effective teacher of computer science at the elementary school level.
2) I have nearly every skill I need to teach computer science well at the elementary school level.
3) I am really good at teaching computer science at the elementary school level. 4) I am effective at making computer science relevant to my students.
52. How much do you agree or disagree with the following statements about computer science? (Scale: Strongly Disagree, Disagree, Somewhat Disagree, Somewhat Agree, Agree, Strongly Agree) 1) I enjoy computer science 2) I am interested in computer science 3) I like doing computer science.
53. What part of computer science do you like the most? Please select only one answer.
1) Designing websites, apps, and/or video games. 2) Computer programming and coding 3) Data analysis 4) Robotics 5) Fixing computer hardware problems 6) Fixing computer software problems 7) Studying how computers work 8) Building computers and other hardware 9) Learning new computer programming languages (e.g. Python, C++, Java) 10) Other (please explain): 54. How much do you agree or disagree with the following statements regarding learning computer science in elementary school? (Scale: Strongly Disagree, Disagree, Somewhat Disagree, Somewhat Agree, Agree, Strongly Agree) 1) Is just as important for students to learn as non-core subjects, like art, music, and foreign language.
1) Explain the logic and reasoning supporting their solutions.
2) Explain why they agree or disagree with the work of other students.
3) Communicate their thought processes to others.

School Grade (Control)
57. What grade(s) do you teach? Please select all that apply.

Essential Program Elements (Control)
First, we'd like to ask you some questions about Achieve3000. As part of the STEM+C Integrated Literacy Block project, you were asked to administer Achieve3000 diagnostic assessments to students. We'd like to know about your students' use of other (non-assessment) Achieve3000 materials and activities (e.g., nonfiction texts from Achieve3000).
58. About how many times did you have your students use Achieve3000 (non-assessment) materials and activities? Number of minutes () 73. Over the past 12 weeks, about how often did you explicitly ask students to do the following? (Scale: Never, Rarely, Sometimes, Often, Always) 1) Answer a question even if they were unsure 2) Try new things even if they might make mistakes 3) Share their ideas even if they were different from others'

Attitudes Toward Interdisciplinary Teaching Practices (Control)
Now we're interested in learning about your attitudes toward interdisciplinary teaching practices. Here is an example of what we mean by "interdisciplinary teaching practices": You teach a 7-week module on North American Regions by: -integrating reading, researching, writing, and presenting about a North American region (ELA); -studying the geography and natural/man-made landmarks of a region (social studies); -researching and studying the climate, vegetation, and natural resources of the region (science); -and using models/simulations to create a map of the region (computer science).
74. How much experience do you have teaching interdisciplinary modules like the one described in the example above? 2) Is more engaging for elementary students than teaching each subject separately.
3) Helps students understand the relevance and applicability of the subjects they are learning.

78.
How much do you agree or disagree with the following statement? 1) Teaching interdisciplinary modules is a feasible instructional model for elementary classrooms.
79. You said you disagree that teaching with interdisciplinary modules is a feasible instructional model for elementary classrooms. What is the main reason you disagreed? Please select only one.
1) I don't have enough time.
2) I lack content knowledge across multiple subject areas.
3) I don't have sufficient access to adequate hardware and/or software. 4) I am not confident teaching with technology. 5) I have difficulty managing my students. 6) I don't have adequate materials. 7) Other (please explain):

Leadership Attitudes Toward CS (Control)
80. We're interested in your perception of your school leaders' attitudes toward computer science. Remember, your answers are confidential and will not be shared with anyone outside of the research team. How much do you agree or disagree with the following statements? My school leaders… 1) Think it's important for all students to learn computer science in elementary school.
2) Believe that learning computer science in elementary school will help students get a good job someday.
3) Communicate the value of computer science to students, parents, teachers, and staff.