Background:

The Statewide Longitudinal Data System (SLDS) conducts internal research and sponsors research in affiliated institutions. This work has supported 9 faculty members and 10 graduate assistants at Montana State University. The Department of Agricultural Economics and Economics conducted a survey on student financial literacy as they enter post secondary education. The Department of Education sponsored research on ACT scores, STEM Careers, High School graduates and locality, and dual enrollment. 

The SLDS has conducted three years of study of the Smarter Balanced Interim Assessment. In addition, the SLDS conducted a study of school trends on the Northern Cheyenne Indian Reservation over the past ten years. This brief will be expanded to other reservations in the coming months. The report titled Culture and Schools was a collaboration between the American Indian Student Achievement (AISA) unit and the SLDS unit. This focus group study involved 45 participants who sought to explain the achievement gap between American Indian students and their peers. Future research will involve a survey of a larger number of schools.


 

OPI Research


 

Circle of Schools - Coordination Between Schools on the Northern Cheyenne Indian Reservation

Consortiums of education institutions develop based on shared challenges and common mission. The Circle of Schools was convened on the Northern Cheyenne Indian Reservation in 2009 to bring together regional community, school, and tribal leadership for the purpose of working together to support student success. The common challenges faced by participants include rural locations and being low income communities. Also common amongst participants, however, is a shared culture and desire to educate all students to their greatest potential. 

Dr. Richard Littlebear (president, Chief Dull Knife College) convened the group of schools which are located on or adjacent to the Northern Cheyenne Indian Reservation in 2009. Working with Ashland Public Schools, Colstrip Public Schools, Lame Deer Schools, Northern Cheyenne Tribal Schools, and St. Labre Schools; Dr. Littlebear aimed to facilitate connections between local schools under the guidance of Chief Dull Knife College faculty. What resulted was a forum where shared challenges could be discussed, and a spirit of collaboration was fostered. One challenge at the forefront for participants was student mobility between partner schools. Montana student data indicates that student mobility is one of the 4 main factors contributing to student dropout risk. Student mobility is persistent in the Northern Cheyenne Indian Reservation, and moderating mobility became a shared goal for the Circle of Schools. To do this, participating schools began encouraging students to transfer at the end of the term and to be clear in the reasons for transferring....Continue Reading Circle of Schools 

Culture & Schools: American Indian Perspectives on the American Indian Student Achievement Gap

On a biennial basis, the OPI produces a quantitative report detailing the American Indian student achievement gap. This report details student outcomes for assessments (SBAC and ACT) and graduation/dropout data points. In order to expand on this report to answer questions about factors influencing the gap, the American Indian Student Achievement (AISA) unit and the Statewide Longitudinal Data Systems unit have launched a three phase study of the achievement and graduation gap between American Indian students that live on- and off-reservation, and their peers throughout the state of Montana.

 

Phase one was a focus group that collected data from 45 stakeholders on-reservation and in off-reservation schools with a high concentration of American Indian students. We found that out-of-school effects were cited nearly as frequently as in school effects. Additionally, the impact of poverty and family and home-life circumstances were more often cited as explanatory factors than issues involving attendance, mobility and dropout. In forthcoming phases (a large-scale survey and interviews with indigenous scholars) these and other areas of inquiry from phase one will be examined in further detail and reported here.

 

Please find below two documents: our executive summary of the focus group findings, and a full technical report.

SBAC Interim Reports

SBAC Interim Report (SY 2018 – 2019)

This report prepared for the Assessment unit in the Montana Office of Public Instruction looks at data from the SBAC Interim Assessment and compares it with data on the SBAC Summative Report. The first analyses compare scores among those students that took the SBAC Interim and Summative. The second analyses compare those scores on the Summative of participating and nonparticipating students. Results were strongly in favor of the use of the SBAC Interim Assessments as a means to gauge student progress mid year, provide feedback to the instructor about each student’s progress, and provide the means to adjust instruction to better meet students needs and help them preform well on the SBAC Summative exam. In ELA, students who took the test scored higher on the Summative, indicating progress between the two time points the tests where taken. In both ELA and math, students that took the SBAC Interim scored significantly higher on the Summative than nonparticipating students. This occurs even though interim test takers were more likely to receive Free or Reduced Lunch.

SBAC Interim Report (SY  2017 – 2018)

This report prepared for the Assessment unit in the Montana Office of Public Instruction compares students scores on the SBAC Interim with the same students scores on the Summative. It looks to the relative effectiveness of each Interim Assessment Block (IAB) and compares it with the summative data found in the SLDS. This research was conducted using all instances a student took a SBAC Instructional Assessment Block (IAB) in the State of Montana. The numbers of students who took at least one IAB is large – 12,491. Most students took the assessment in grade 3-5. Two grade levels stand out as achieving especially meaningful gains for each IAB. The majority of the third grade IABs showed that average student score improved significantly in 5 of the 7 IAB tests.  Grade 6 also experienced meaningful positive changes within 3 of 5 IABs measured. The range for the scale scores is 2280 – 2680. The takeaway from this research is that average student scores show significant gains for those students taking an Interim assessment. This could be due to the student having more experience testing or the teacher using the test to guide instruction, or both.

 


 

 

Partner Research

 


 

ACT Outcomes

Brazill, S. (2019). The Relationship of Gender, School Attendance, and Grade Level with ACT English and Composite Test Scores. In G. Marks (Ed.), Proceedings of Global Learn 2019-Global Conference on Learning and Technology (pp. 64-69). Princeton-Mercer, New Jersey: Association for the Advancement of Computing in Education (AACE). Retrieved October 6, 2020 from https://www.learntechlib.org/primary/p/210291/ [learntechlib.org].

Factors that predict ACT science scores from a multicultural perspective. Educational Research: Theory and Practice, 30(2), 1-16.

This study investigated predictors for ACT Science scores, a test used by many universities to rank applicants. This study utilized quantitative research methods using the Montana Office of Public Instruction’s GEMS (Growth and Enhancement of Montana Students) data set. All advanced statistical analysis was conducted using Stata software IC/15. This research is significant for increasing the representation of under-represented groups in STEM education because it helps clarify three important relationships: (1) How well do gender, race, and meal status predict 11th grade ACT Science scores; (2) How well does school size predict 11th grade ACT Science scores while controlling for gender, race, and meal status; and (3) How well does high school GPA predict 11th grade ACT Science scores while controlling for gender, race, meal status, and school size.

This manuscript utilized quantitative research methods to analyze the Montana Office of Public Instruction’s GEMS (Growth and Enhancement of Montana Students) dataset. This research is important because it helps clarify three important relationships: (1) Is there a statistically significant difference in the average 11th grade English scores between males and females; (2) Does the number of days present at school influence students’ 11th grade ACT composite scores; and (3) Do individual students’ ACT English scores improve from 11th grade to 12th grade.

Brazill, S. C. (2020). Race and gender factors in ACT English and composite scores. Educational Research: Theory and Practice, 32(1), 17-28.

This study utilized quantitative research methods using the Montana Office of Public Instruction’s Growth and Enhancement of Montana Students (GEMS) data set. This quantitative research is important because it provides insight on the following relationships (1) Is there a statistically significant difference among students from different races on their mean 11th grade ACT English scores?; (2) Is there an interaction between gender and race on students’ mean 11th grade ACT English scores?; and (3) Is there a statistically significant difference among American Indian students mean ACT composite score from their Junior to their Senior year.

Obery, A., Sletten, J., Vallor, R. & Schmitt-Wilson, S. (2020) Data Driven Decision Making in Teacher Education: Perceptions of Pre-Service Teachers and Faculty Who Teach Them, Action in Teacher Education, DOI: 10.1080/01626620.2020.1762139 [doi.org]

Integrating data driven decision making into teacher preparation programs has been called for by several researchers in order to help lessen the barriers teachers face in using data in their teaching. This mixed methods study examined the perceptions of data use in education by pre-service teachers (N = 112) and perceived knowledge and skills required for successful use from both pre-service teachers (N = 6) and the faculty who educate them (N = 5). Results have implications for teacher preparation programs considering data literacy integration and for researchers aiming to understand the best ways to encourage data driven decision-making skills and knowledge in pre-service teachers.

Better Borrowing for Students - How State Mandated Financial Education Drives College Financing Behavior

Paper Title: The Effects of State Mandated Financial Education on College Financial Behavior

Authors: Christina Stoddard, PhD & Carly Urban, PhD

Affiliation: Department of Economics, Montana State University

Funded by: Montana Office of Public Instruction and the National Endowment for Financial Education

 

This paper examines the effect of high school personal finance graduation requirements on student decisions about how to finance college. Personal finance requirements exist in many different forms in twenty-five states. The authors conduct an analysis to evaluate the factors that affect financial decision making during their freshman year. They compare students that are in states with those requirements to students in states where financial education is not required for graduation. Montana is examined in a case study as a state which does not have state required financial education. The analyses are informed by data from the National Post-Secondary Student Aid Study (NPSAS), Current Population Survey (CPS), Integrated Post-Secondary Education Data Systems (IPEDS), and data from the Montana Office of Public Instruction and the Montana University System.

 

The analysis uses data from 25 states. This approach is used “to compare incoming freshman at four year institutions from states with personal finance requirement before and after implementing the requirement to comparable students whose states lack a mandate” (Stoddard & Urban,2018, p. 2).

 

Stoddard & Urban (2018, p.1) suggest the following findings from their study:

  • Financial education graduation requirements shift students from high-cost to low-cost financing behaviors.
  • Requirements increase federal aid applications and acceptance of low cost federal student loans.
  • Requirements decrease the likelihood of students holding credit card balances.
  • Students from less affluent family backgrounds further reduce their likelihoods of working while enrolled.
  • Borrowers from more affluent backgrounds reduce private loan amounts.
  • The mandates do not change college attendance or choice of institution type.

The data from Montana suggest:

  • There are no statistical differences across students in schools that do and do not offer personal finance course (Stoddard & Urban, 2018, p. 14)
  • There are no visible patterns in schools offering and not offering financial education based on their geography or distance from main highways in the state (Ibid).
  • School choices to add a personal finance course are idiosyncratic (Stoddard & Urban, 2018, p. 32).
  • There are no statistical differences in having subsidized Stafford loans, having unsubsidized Stafford loans or having a grant (Ibid).

There are several implications to the study. Personal finance graduation requirements improve student financial behaviors in college. The benefits may extend beyond college financing decision. The report may be downloaded from https://gems.opi.mt.gov/.

Choice of STEM Major Research

"College Enrollment and STEM Major Choice in a Rural State: A Statewide Examination of Recent High School Cohorts, examined both institution type enrollment (2-year v. 4-year) and college major selection (STEM v. non-STEM) using the GEMS random data set of high school cohorts graduating in 2013-2017. The data set was released under a cooperation among the Office of the Commissioner of Higher Education, the OPI, and Dr. Fenqjen Luo at the Department of Education, MSU. We are only reporting findings for students who stayed in Montana for college. The study examined 3,119 students and found that Montana students are more likely to enroll into a 4-year institution than a 2-year institution (79.6% v. 20.4%). Also, students who enrolled at a 4-year institution are more likely to consider a STEM major than students at a 2-year institution. When examining institution type, significant explanatory variables included high school GPA, ACT STEM score, ACT English score, free/reduced-price lunch participation, and race/ethnicity. An interesting finding showed that American Indian/Alaska Native students were more likely to enroll into a 4-year institution than White students. When examining major choice, only high school GPA, ACT STEM score, gender, and institution type were significant predictors of selecting a major in a STEM field. Specifically, male students were over three times as likely to select a major in a STEM field when compared to their female counterparts. The authors also discuss limitations and recommendations for future research. The study is currently under review in Theory & Practice in Rural Education. 

This research was conducted at MSU by two doctoral students, Que N. Tran and Monte Meyerink, and two faculty members, Dr. Alexandra Aylward and Dr. Fenqjen Luo. Additionally, we acknowledge the excellent support from Dr. Robin Clausen at OPI and OCHE during this research project. This research was funded by the grant, "Collaborative for Continuous Improvement in Education: Montana GEMS Data and the K-16 Pathway," from the Montana Office of Public Instruction for the College of Education, Health, and Human Development, MSU during the academic years 2017-2019 (Dr. Tricia Seifert, Principal Investigator).