JOURNAL for K-12 EDUCATIONAL RESEARCH Volume 3 I Issue 1 a publication of the CENTER for K-12 EDUCATIONAL LEADERSHIP
PROGRAM FEATURES • Degree completion in four years while working full-time • Direct application to current K-12 issues and trends • Lifelong relationships through the cohort model • Research that will impact your school or district • Three weekend classes per semester • Summer Symposiums in Austin and Washington, D.C. • Tuition competitively priced and financial aid available For more information, contact: Dr. Neil Dugger I 214.333.5202 I neil@dbu.edu I www.dbu.edu/doctoral/edd Dallas Baptist University I College of Education I 3000 Mountain Creek Parkway, Dallas, TX 75211 ED.D. IN EDUCATIONAL LEADERSHIP K-12 Equipping Servant Leaders for the 21st Century The Ed.D. in Educational Leadership K-12 program emphasizes a practical approach to leadership development utilizing the servant leadership model. This degree program is for individuals who wish to have a transformational impact on their K-12 setting through their ability to integrate faith and learning. The program is both academically rigorous and practical in design. EARN YOUR DOCTORATE AT DBU
Journal of K-12 Educational Research 1 TABLE OF CONTENTS GREETINGS FROM THE DIRECTOR. ..........................................................................................................................................3 AWORD FROM THE EDITOR.......................................................................................................................................................4 SEARCHING FOR DEMOCRACY IN THE MARKET: A PHENOMENOLOGICAL STUDY OF TEXAS SUPERINTENDENTS NAVIGATING SCHOOL CHOICE REFORM.......................................................................................5 Paul Cook A STUDY OF TEACHERS’ PERCEPTIONS OF THE TEXAS TEACHER EVALUATION AND SUPPORT SYSTEM...............................................................................................................................................................11 Julia M. Hyman POSITIVE BEHAVIOR INTERVENTION SUPPORT SYSTEMS AND THE DISPROPORTIONALITY OF SUBJECTIVE DISCIPLINARY REFERRALS FOR BLACK STUDENTS. ................................................................................18 Alisha M. Brown DIFFERENCES IN EARLY LITERACY ACHIEVEMENT FOR ENGLISH LANGUAGE LEARNERS: ANALYZING PERFORMANCE BETWEEN TRANSITIONAL AND DUAL LANGUAGE BILINGUAL PROGRAMS..........................23 Lori DeAnn Rapp THE EFFECT OF AN INTENSIVE SUMMER READING INTERVENTION PROGRAM ON ENGLISH LEARNER STATE ASSESSMENT SCORES.....................................................................................................................................................30 Tim E. Baxter THE IMPACT OF LEADERSHIP BEHAVIORS ON TEACHER ENGAGEMENT................................................................37 Jennifer Wilson INSTRUCTIONAL LEADERSHIP DEVELOPMENT IN ONE NORTH TEXAS DISTRICT................................................43 Shawna Miller PRINCIPAL LEADERSHIP DEVELOPMENT PLANS AND THE PERCEIVED IMPACT ON SCHOOL CULTURE.......................................................................................................................................................................50 Le'Ann D. Downs TRANSFORMATIONAL SCHOOL DISTRICTS: BEST PRACTICES FOR RECRUITING AND RETAINING TEACHERS...............................................................................................................................................................57 James Hill JOURNALOF K-12 EDUCATIONALRESEARCH
2 JOURNAL OF K-12 EDUCATIONAL RESEARCH PUBLISHING INFORMATION EDITOR Sharon Lee, Ph.D. ASSISTANT EDITOR Aubra Bulin, Ed.D. TABLE OF CONTENTS ART DIRECTOR Kadra Fant LAYOUT Shea Bush FACTORS AFFECTING JOB SATISFACTION AND TEACHER RETENTION FOR NORTH TEXAS SECONDARY SCIENCE TEACHERS....................................................................................................................................................................63 Christopher Michael Miller NEW TEACHER AND MENTOR PERCEPTIONS OF MENTORING AND TEACHER RETENTION IN AN AFFLUENT DISTRICT..........................................................................................................................................................................................69 Jaime Callahan LEADING THE MILLENNIAL GENERATION: AN INVESTIGATION OF LEADERSHIP PRACTICES THAT ENCOURAGE AND SUPPORT EDUCATION'S YOUNGEST TEACHERS..........................................................................74 Julie DeLeon
Journal of K-12 Educational Research 3 Neil Dugger, Ed.D. GREETINGS FROM THE DIRECTOR Journal of K-12 Educational Research 2019, VOL. 3, ISSUE 1 www.dbu.edu/doctoral/edd Greetings from Dallas Baptist University! This is the third volume of the research journal produced by the Ed.D. in Educational Leadership K-12 program, and all the articles were written by our doctoral graduates. We hope you will find the articles timely, interesting, and resourceful. In order to make solid decisions in our schools, we must have access to good research and data. The Journal is one source for you to turn to and find the real facts. These articles cover the spectrum of academic issues and are just a small sampling of the 20+ graduates we have each year. Contact information is provided, so if you would like to review entire studies, the authors will be happy to provide them for you. DBU graduates are trained to be “servant leaders who are transforming the world,” and they are having a significant impact in districts across the DFW area and beyond. We need school leaders who are pure in their motives and always striving to meet the needs of every student. If our graduates are representative of this higher standard, the future of education in Texas is great! If you know of a young educator who fits this model and has a great future leading our schools, please send him/her our way to see if the DBU doctoral program is the right fit. It is a practitioner’s degree, and classes meet three weekends per semester. Most students finish in less than four years, and their dissertation (treatise) topic is designed to have an impact on their school system. For more information about the degree, please contact me at neil@dbu.edu. Thank you for the critical role you play in the education of the children in our state and nation—there is no greater calling. Sincerely, Neil Dugger, Ed.D. Director, Ed.D. in Educational Leadership K-12
4 Sharon Lee, Ph.D. AWORD FROM THE EDITOR Dallas Baptist University is proud to present the third volume of the Journal of K-12 Educational Research. The articles in this issue are presented by a selection of recent graduates of the Doctorate in Educational Leadership K-12, a practitioner’s degree which highlights skills needed to lead districts in North Texas and beyond. Each article is a summary of the data gathered for their individual research projects, and each provides answers to questions of local interest. In some cases, the authors provided state-wide data of interest to the broader educational community. Other authors gathered data from a smaller data set or a single district. Whether state-wide or local data, we believe the answers may be applicable to many educational settings. In the DBU program, student researchers are encouraged to look for immediate and site-based solutions that could easily be transferrable to issues that are wide and far-reaching. In this issue, you will find articles that have a national impact such as the study by Dr. Paul Cook who interviewed superintendents about their perceptions of the current state of school competition through a discussion of the market and democracy metaphors. Some of the research was conducted in multiple districts on topics of local interest. Dr. Julia Hyman surveyed teachers about the T-TESS system of teacher evaluation, and Dr. Alisha Brown explored Positive Behavior Interventions and Support (PBIS). An important topic for most districts in North Texas is the education of language learners. Dr. Lori Rapp executed a state-wide review of various dual language programs in the early elementary grades and provides some data about effectiveness of those programs. Dr. Tim Baxter explored special programs for secondary language learners. Leadership development is a topic that is of great interest to both campus- and district-level leaders. Dr. Jennifer Wilson explored perceptions of district leadership style from teachers in private schools. Dr. Shawna Miller documented the outcomes of a district-wide leadership cohort program. Dr. Le’Ann Downs shared the impact of leadership development plans on district climate and culture. Recruiting and keeping teachers is an issue in many North Texas districts. Dr. James Hill interviewed HR directors to explore some strategies used by districts to retain teachers. Dr. Chris Miller surveyed science teachers in North Texas about their decisions to stay in the field of teaching. Both studies provide valuable information about teacher recruitment. Once teachers are hired in districts, there is still concern about how to keep them. Dr. Jamie Callahan explored a new teacher mentoring program in her district, while Dr. Julie DeLeon documents some specific strategies for retaining the youngest and newest members of the teaching profession. These studies provide detailed information about teacher retention. DBU’s Ed.D. K-12 program is based on the Biblical servant leader model of putting the needs of others first. Students explore problems that have immediate concern in their districts and are encouraged to bring answers to those districts. The articles included in this issue and the final treatises upon which they were based have the potential to empower educational leaders and classroom educators, while making a difference for students in the classrooms throughout the North Texas region and beyond. Sharon Lee, Ph.D. Director of Research in K-12 Education Dallas Baptist University Editor: Journal of K-12 Educational Research Journal of K-12 Educational Research 2019, VOL. 3, ISSUE 1 www.dbu.edu/doctoral/edd
Journal of K-12 Educational Research 5 SEARCHING FOR DEMOCRACY IN THE MARKET: A PHENOMENOLOGICAL STUDY OF TEXAS SUPERINTENDENTS NAVIGATING SCHOOL CHOICE REFORM Paul Cook, Ed.D. Introduction The American educational marketplace is more dynamic today and nuanced than it has ever been (EdChoice.org, 2017). Charter schools have completely replaced public schools in New Orleans and almost replaced them in Detroit. Florida and Indiana have increased their voucher programs at the state level by more than 400% while states like Arizona and Nevada have passed legislation to make vouchers available for every student in the state (Black, 2017). Conversations about the continued diversification of educational offerings in the United States are not new. In the 1990s many educational researchers were busy discussing this changing landscape including Peter Cookson (1994). Cookson, an educational sociologist, posited that there was an ongoing shift in American ideologies of education reform. This shift could be attributed to the growth and proliferation of market-based reform practices. Cookson saw the growth of school-choice-based initiatives in 1994 as an indicator that American sentiment towards communal accountability and national identity was giving way to ideas about American individualism and personal freedoms. Cookson suggested that a metaphorical framework could be used to describe this change. The metaphors he selected are that of democracy and the market. Purpose of the Study The purpose of the current study was to assess the phenomenological experiences of public school superintendents navigating these market-based reform initiatives. With their respective backgrounds, superintendents have almost universally experienced significant historical changes within their career which have a formative effect on their beliefs about public education and their assessment of its current efficacy in relationship to its past iterations. Acknowledging the myriad voices in the school change conversation, and recognizing the lack of phenomenological data from superintendents, created the purpose for the current study. The researcher chose to focus on the experiences of superintendents in the North Texas area with two competing reform metaphors, the market and democracy. These metaphors are further understood through the addition of eight respective dimensions, four for each metaphor. Review of Literature According to Cookson, these two metaphors are in competition with each other and represent a very real tension between the ideological camps of educational reformers. In his framework, market reformers view educational landscapes primarily in terms of transactions and recognize the “primacy and efficacy of consumership as a way of life” (Cookson, 1994, p. 99). Opposite the market is the metaphor of democracy in which Cookson described the aim as the “primacy and efficacy of citizenship as a way of life” (Cookson, 1994, p. 99). Markowitz (2001) offered a similar metaphorical framework for evaluating educational reform initiatives. While her work dealt specifically with “Deweyan democracy” and “Charter theory,” the characteristics of each metaphor are easily understood within the context of Cookson’s reform theories. Markowitz identified four distinct characterizations for both Deweyan democracy and Charter theory. Deweyan democracy is identified by dimensions labeled Association, Connection, Communication, and Growth. Journal of K-12 Educational Research 2019, VOL. 3, ISSUE 1 www.dbu.edu/doctoral/edd
6 Charter theory is identified by dimensions labeled Competition, Autonomy, Choice, and Productivity. Hastings (1999) set the stage for understanding the evolution of reform ideologies in public education in America. His work highlighted a shift from market-based reformers, to common school reformers and large-scale public education systems, back to market-based reformers like Milton Friedman (1982). Markowitz (2001) identified John Dewey as a contrast to market based reform ideologies. Her work highlighted four dimensions of what she referred to as “Deweyan democracy.” The first of these dimensions is association. Association according to Markowitz, (2001) speaks to the notion of identity in a society. Dewey’s second category of democracy, connection, is inexorably connected to the idea of association. While association provides the context for life in community, connection speaks to the interactions within that community and specifically to the attention to synergistic or mutually beneficial interactions within that community. The third category of Deweyan democracy, communication, draws attention to the manner in which the connections within society take place. The final component of Deweyan democracy is the category of growth. Growth, according to Dewey, is the measure of the efficacy of a society. Opposite Deweyan democracy is market theory. One of the chief proponents of market theory in the United States was Milton Friedman. As an opponent of the American public education system, Friedman (1982) identified a primary point of contention, the government monopoly. In line with his other economic principles, Friedman believed that in terms of economy, the monopoly-controlled education market represented the least effective model for school reform. The work of Friedman to advance market-based reform efforts continued into the 1990s with reformers like Chubb and Moe (1990) and Lieberman (1995). This emphasis on market reform is characterized by Markowitz (2001) through what she calls “charter theory.” Charter theory contains four key dimensions. Markowitz’s (2001) first characteristic of the charter theory is competition. Competition, as identified by free market economists, is a vehicle for achieving in economics what is commonly known as “pareto optimality.” This optimality can best be understood as a sense of equilibrium in educational markets where all participants benefit to the greatest extent possible. The second characteristic of Markowitz’s (2001) charter theory framework is that of autonomy/accountability. In line with Friedman’s (1982) point of view, existing government-run educational systems are characterized as bloated, bureaucratic, inflexible, and immovable. Competitive markets would again answer this difficulty by providing educators who had greater flexibility, autonomy, and responsiveness than their traditional, government-run, counterparts. This idea was bolstered in Friedman’s lifetime by the birth of educational alternatives like charter schools. The third characterization within Markowitz’s (2001) charter theory is that of choice. Friedman’s notions of free market economics as a method for school reform were founded upon a concept in microeconomics known as rational choice theory (Kalmar, 2014). According to Kalmar, (2014) rational choice theory is the lynchpin under which the philosophy of free market economics operates in the educational realm. This theory suggests that the consumer will, with a degree of predictability, regularly select for themselves amongst their options that which maximizes their own personal utility. The final characterization Markowitz (2001) offers for charter theory is that of productivity. The field of economics places a high regard on market elements like inputs and outputs and places value on those elements through economic terms like productivity. This emphasis on efficiency and outputs is a hallmark of market-based reform theories. Research Design The primary tool used for collection of research data in the current study was semi-structured interview questions used to gather qualitative data that could be analyzed for themes and patterns. The current study employed maximum variation sampling with a population that included superintendents from eight of the nine Texas Education Agency (TEA) categories of school districts in the state of Texas. The charter school category was intentionally removed as the defining attributes of the charter school model did not align with the Markowitz democracy framework (Markowitz, 2001). The researcher explored the Paul Cook, Ed.D.
Journal of K-12 Educational Research 7 perceptions of school district superintendents on eight different themes. These themes were drawn from the work of Cookson (1994), Markowitz (2001), and Engel (2000). These eight themes represent two different interpretive metaphors each containing four components. Procedures and Data Analysis Semi-structured interviews were conducted utilizing an interview protocol meant to give participants an opportunity to reflect on each metaphor dimension individually. This qualitative data was then analyzed utilizing NVivo 12 to identify major category nodes, parent nodes, child nodes, and theme nodes. These nodes and findings were used to answer the following research questions. (RQ1) How do Texas public school superintendents align with two different educational reform metaphors (democracy and market)? (RQ1A) How do Texas public school superintendents align with the market metaphor dimension of autonomy? (RQ1B) How do Texas public school superintendents align with the market metaphor dimension of choice? (RQ1C) How do Texas public school superintendents align with the market metaphor dimension of competition? (RQ1D) How do Texas public school superintendents align with the market metaphor dimension of productivity? (RQ1E) How do Texas public school superintendents align with the democracy metaphor dimension of association? (RQ1F) How do Texas public school superintendents align with the democracy metaphor dimension of connection? (RQ1G) How do Texas public school superintendents align with the democracy metaphor dimension of communication? (RQ1H) How do Texas public school superintendents align with the democracy metaphor dimension of growth? Results The Market dimension Autonomy found respondents identified positively with the dimension with 57% of their comments, reflecting a desire for increased levels of autonomy in the relationship between school districts and the relationship between managerial layers within the district. Respondents identified negatively with the dimension of Autonomy 43% of the time, representing a 14% disparity between positive and negative associations. Negative associations surrounding the dimension of Autonomy centered primarily around managerial leadership within school districts and a desire to limit Autonomy for subordinates at the classroom level. The Market dimension Choice was viewed positively by superintendents. In terms of percentages, 70% of superintendent responses favored the Market dimension of Choice while 30% did not. This represented a 40% disparity between responses with the majority of participant references responding favorably to the dimension of Choice. Two commonalities among participant responses pertained to the ability of larger school districts to provide adequate choices and a desire for all districts to offer additional choices to students and families in the future. Smaller districts envisioned themselves as handicapped in terms of educational choice offerings while all districts expressed an interest in expanding choice options in the future. The Market dimension Competition did not demonstrate a favorable alignment with superintendent responses. The corresponding percentages were 27% of responses supportive of the Competition dimension with 73% of responses demonstrating opposition to Competition in education. Participants tended to differentiate between student competition, campus competition, and competition for students among educational providers. While participants generally favored Competition among students and traditional ISD campuses, participants were generally opposed to charter schools, citing concerns about funding, admission standards, financial oversight, and comparability with traditional public schools. The Market dimension Productivity was the third market dimension with which superintendents aligned favorably. The corresponding percentages were 60% alignment and 40% non-alignment. This dimension emphasizes the goal of education in primarily economic and financial terms exclusively. Superintendents generally struggled to outline educational outcomes that were not primarily oriented around future productivity of students: earning potential, tax paying abilities, and job attainment. The Democracy dimension Association showed high levels of alignment with superintendents. The percentages included 98% alignment and only 2% non-alignment. The researcher
8 categorized response data into several sub nodes that helped identify how superintendents view the idea of Association. These nodes are Inclusivity and Identity. The Inclusivity node contains participant responses that identify Association particularly in terms of involving stakeholders in the work they do. The Identity node contains responses in which superintendents speak to the dimension of Association as the establishment of a shared or communal identity that includes the school. The Democracy dimension Communication showed strong alignment with superintendents. With 57 references, the Communication dimension represents the largest of all eight dimensions suggesting that Communication is a topic that is especially pertinent to the superintendent. With all 57 references indicating an alignment with the Communication dimension, the researcher chose to further categorize participant responses in order to yield deeper insight into the superintendents' perceptions of Communication. The researcher created sub nodes Collaboration, Engagement, Internal, Responsiveness, and Transparency. These nodes reflected the vast majority of intent for Communication among superintendents. The Democracy dimension Connection showed marginal alignment with superintendent responses. This dimension did require the creation of additional sub nodes to explore the alignment of responses. The purpose of the sub nodes was to differentiate between connections that only benefitted the organization, versus connections that benefitted a variety of stakeholders. The contrast between these two types of connections helped further identify superintendent alignment with the dimension of Connection. Respectively, the Pro and Against categories correspond to the following percentages, 57% and 43%, representing a 14% disparity in responses. The Democracy dimension Growth showed significant levels of alignment with superintendent responses. This dimension necessitated the creation of sub nodes for further exploration of respondent data. These sub nodes differentiate between Growth as a purely organizational effort and growth for the sake of internal and external shareholders. This contrast provided the Pro and Against dynamic necessary for alignment analysis. The breadth and frequency of responses in the Pro sub node indicate that, in general, superintendents characterize Growth in very diverse terms as opposed to the singular nature of the Against sub node. This diversity in definition falls into alignment with ideals of growth in a Deweyan democracy. All of the metaphor dimensions are presented in Table 1 with percentages of Pro and Against. Implications Conservative educational researchers in the 1990s expressed their concern about dominant reform metaphors in the realm of public education (Cookson, 1994; Engel, 2000; Henig, 1994; Tyack & Cuban, 1994). These concerns revolved around the notion that the American public education system would continue to embrace market ideals and practices in search of greater efficacy and public satisfaction. This pursuit would come, in the minds of these researchers, at significant cost, namely the loss of democracy in the public education arena. Contemporary researchers and advocates for traditional public education express similar concern in their research (Black, 2017; Ravitch, 2010, 2014; Vasquez Heilig, Williams, McNeil, & Lee, 2011). The current study highlights that despite 25 years of criticism by conservative reformers, market-based practices and ideologies are ever present and expanding among North Texas superintendents. The findings of the current study complement previous research frequency of responses in the Pro sub node indicate that in general superintendents characterize Growth in very diverse terms as opposed to the singu r natu e of the Ag inst sub node. This diversity in definition falls into alignment with ideals of growth in a Deweyan Democracy. All of the metaphor dimensions are presented in Table 1 with percentages of Pro and Against. Table 1. Metaphor Dimension Alignment Table 1. Metaphor Dimension Alignment Paul Cook, Ed.D.
Journal of K-12 Educational Research 9 in the field of educational reform and indicate that the concerns of conservative educational reformers voiced in the 1990s remain valid if not more pressing today. The findings of the current study are in alignment with the positions expressed by previous researchers both historically (Cookson, 1994; Engel, 2000; Henig, 1994; Tyack & Cuban, 1994) as well as currently (Black, 2017; Ravitch, 2014; Vasquez Heilig, et al., 2011). The current study highlights the prominence and prevalence of both metaphors in the experiences of current sitting superintendents impacted by these metaphors. The current study also emphasizes a need for continued dialogue and analysis of reform metaphors in the realm of public education. Previous researchers directed their efforts toward the articulation of these metaphors and their presentation to educational reformers. In the decades since, the current study indicates that these reform metaphors have continued to grow and currently are represented in the operational realities of sitting superintendents. In the theoretical sense, the continued study of these metaphors may yield new insights into how to navigate future reform efforts as well as identify sociological trends that impact public education. Policymakers, educational providers, and communities are required to participate in reform measures often without consideration given to the philosophical drivers of these reform policies. The further development of the terminology and metaphorical framework for understanding these reforms would enable participants to make more informed decisions as they navigate an already complex landscape of educational reform. References Black, D. W. (2017, October 24). Preferencing educational choice: The constitutional limits. Cornell Law Review, 103, 1359-1430. Retrieved from http://cornelllawreview.org/ files/2018/12/Blackfinal.pdf Chubb, J. E., & Moe, T. M. (1990). Politics, markets, and America’s schools. Washington, DC: Brookings Institution. Cookson, P. W. (1994). School choice: The struggle for the soul of American education. New Haven, CT: Yale University Press. Engel, M. (2000). The struggle for control of public education: Market ideology vs. democratic values. Philadelphia, PA: Temple University Press. Friedman, M. (1982). Capitalism and freedom (2nd ed.). Chicago, IL: University of Chicago Press. Hastings, H. J. (1999). Making choices about school choice: A study of the legal and ideological issues of education vouchers in the United States (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (AAT3079561) Henig, J. (1994). Rethinking school choice: Limits of the market metaphor. Princeton, NJ: Princeton University Press. Kalmar, W. F., Jr. (2014). School choice and the decision-making of school leaders (Doctoral dissertation). (Order No. 10586561) Retrieved from ProQuest Dissertations & Theses Global. Lieberman, M. (1995). Public education: An autopsy. Cambridge, MA: Harvard University Press. Markowitz, M. C. (2001). A critique of charter school theory rom a Deweyan perspective on democracy and education (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (UMI No. 3018516). Ravitch, D. (2010). The death and life of the great American school system: How testing and choice are undermining education. New York, NY: Basic Books. Ravitch, D. (2014). Reign of error: The hoax of the privatization movement and the danger to America’s public schools. New York, NY: Vintage Books. Tyack, D., & Cuban, L. (1994). Tinkering toward utopia: A century of public school reform . Cambridge, MA: Harvard University Press. Vasquez Heilig, J., Williams, A., McNeil, L., & Lee, C. (2011). Is choice a panacea? An analysis of Black secondary student attrition from KIPP, other private charters and urban districts. Berkeley Review of Education, 2 (2), 153-178.
10 About the Author Dr. Paul Cook serves as Assistant Principal in the Hurst-Euless-Bedford Independent School District. Previous positions held include administration and teaching in Birdville ISD and Arlington ISD. He holds a B.A. in Church Music from Dallas Christian College, Master's in Business Administration from the University of Texas at Arlington, and a Doctor of Education in Educational Leadership K-12 from Dallas Baptist University. He can be reached at paul.cook@outlook.com. Paul Cook, Ed.D. Please use the following template when submitting your b Educational Research. Dr. Paul Cook serves as Assistant Principal in the Hurst-Euless-Bed ositi ns held include administration and teaching in Birdville ISD Music from Dallas Christian College, Masters in Business Adminis and a Doctor of Education Degree in Educational Leadership K-12 f at paul.cook@outlook.com.
Journal of K-12 Educational Research 11 A STUDY OF TEACHERS’ PERCEPTIONS OF THE TEXAS TEACHER EVALUATION AND SUPPORT SYSTEM Julia M. Hyman, Ed.D. Introduction A perception exists that teaching quality and student performance are closely related. If students have exposure to better teaching, it seems logical that learning and student performance exhibit a corresponding improvement (Tucker & Stronge, 2005; Wright, Horn, & Sanders, 1997). Recent efforts to reform teacher evaluation systems have a foundation in this school of thought. Federal and state laws have guided the redesign of teacher evaluation systems with the goal of improving teaching quality. These new teacher evaluation systems have been based on teaching standards and have been designed to measure teacher effectiveness through multiple measures such as competencybased observations and student growth measures. Evaluation systems presume that teachers will use data generated by these measures to engage in professional learning and inform instructional practices. The recursive nature of this evaluation process is built upon continuous improvement practices designed to inform the efforts of teachers and school leaders in pursuit of improved student learning (Danielson, 2007; Darling-Hammond, 1999; Herlihy et al., 2014; Marzano, Frontier, & Livingston, 2011; Milanowski, 2011; Minnici, 2014; Popham, 2013; Weisberg, Sexton, Mulhern, & Keeling, 2009). The Texas Teacher Evaluation and Support System (T-TESS), implemented in 2016-17, was designed to engage Texas teachers in an evaluation system, which included goal setting, analysis of observation and student growth data, self-reflection, collaborative coaching, and professional learning. The purpose of T-TESS was to provide teachers and districts the tools necessary to engage in continuous improvement efforts which would result in improved quality of teaching and student performance (Texas Education Agency, 2016). Summary of Study Educational reforms can be change agents that impact organizational systems and contribute to personal concerns on the part of teachers (Van der Vegt, Smyth, & Vandenberghe, 2001). Negative or positive perceptions of change often influence teacher commitments and actions regarding the implementation of systemic reform—ultimately impacting the outcome of efforts (Claudet, 1999). To obtain the intended outcomes of teacher evaluation reform, it is necessary that teachers believe there is a valid measure of effective teaching, ratings are reliable, feedback is useful, and there is a credible purpose for participation. If this does not hold true, teachers are likely to participate for purposes of compliance rather than with the intent to change instructional practices (Eisenhart, Cuthbert, Shrum, & Harding, 1988; Hampton, 2016; Heneman & Milanowski, 2003; Hill & Grossman, 2013; Minnicci, 2014; Nordin, 2014; Papay, 2012; Reddy et al., 2017; Stecker et al., 2018; Thomson, 2013; Van der Vegt et al., 2001). The purpose of the current study was to measure teachers’ perceptions in these key areas: 1) the validity of the T-TESS rubric descriptions of effective teaching, 2) the reliability of evaluators to accurately evaluate teacher performance, and 3) the usefulness of feedback data to influence instructional practice. Additionally, the current study set out to measure teachers’ understanding of the purposes for T-TESS and their willingness to participate in the evaluation process with the intent to change instructional practice. Journal of K-12 Educational Research 2019, VOL. 3, ISSUE 1 www.dbu.edu/doctoral/edd
12 The research questions that framed this current study were the following: Research Question 1 (RQ1) Do teachers perceive the T-TESS rubrics as valid descriptors of effective teaching? Research Question 2 (RQ2) Do teachers perceive the T-TESS ratings as reliable measures of effective teaching? Research Question 3 (RQ3) Do teachers perceive the feedback given from evaluators using the T-TESS rubrics as effective in influencing teaching practice? Research Question 4 (RQ4) What are teachers’ overall perceptions of the purpose of T-TESS? Research Question 5 (RQ5) How do teachers report using T-TESS to improve instructional practice? Research Question 6 (RQ6) What are teachers’ perceptions of the use of student growth measures in the T-TESS teacher evaluation system? Literature Overview Use of T-TESS to inform instructional practice and improve student learning rests on the perceptions and resulting actions of teachers. Perceptions of the face and social validity of the competency-based observation rubrics and inclusion of student growth measures may influence the willingness of teachers to engage in the system with the intent to change practice (Cohen & Goldhaber, 2016; Hargreaves, 2005; Herlihy et al., 2014; Hampton, 2016; Holmes, 2016; Minnici, 2014; Papay, 2012; Reddy et al., 2017). The ability of the evaluation system to produce reliable ratings across evaluators and observations is dependent upon the design of the system and the competency of the evaluators (Minnici, 2014; Nordin, 2014; Papay, 2012; Strong, Gargani, & Hacifazlioglu, 2011; Weisberg et al., 2009). Expanding the number of raters and the frequency of observations may, with purposeful training, increase the reliability of ratings for all teachers, including those teaching specialized groups of students (Cohen & Goldhaber, 2016; Herlihy et al., 2014; Hill, Charalambos, & Kraft, 2012; Hill & Grossman, 2013; Jones & Brownell, 2014; Jones, Buzick, & Turkan, 2013; Milanowski, 2011; Weisberg et al., 2009). Perception of the usefulness of feedback to inform professional learning and guide change in instructional practice may be dependent upon systemic functionality factors. These factors may influence a teachers’ approach to use of data for its intended purpose, limited application, or deliberate abandonment of use. Systemic functionality may also limit the ability of the district to construct personalized professional learning stemming from evaluation data (Ikemoto & Marsh, 2007; Nordin, 2014; Stecker et al., 2018). The history of teacher evaluation in Texas demonstrated a pattern of optimism and positive perception in the initial stages of implementation. However, problems with systemic functionality, perceived gaps in fidelity to system design, and distrust in the ability of the system to accomplish the goal of improved teaching quality over time resulted in the growth of negative perception and attitude on the part of teachers (Davis, 2013; Davis-Frost, 2000; Ettema, Sengupta, & Kress, 2014; Rigsby, 2014; TEA, 1991). Teacher evaluation systems are complex, and successful use of such systems to improve teaching quality is dependent upon the participation of both teachers and school leaders in continuous improvement efforts. If such continuous improvement efforts are ignored, it is likely that teachers will develop increasingly negative perceptions of T-TESS and the earlier pattern of teacher evaluation in Texas will be replicated (Heneman & Milanowski, 2003; Papay, 2012; Minnici, 2014; Stecker et al., 2018). Findings The basic convergent mixed-methodology of the current study established the concurrent collection and analysis of quantitative and qualitative data. Quantitative analysis of survey responses indicated that teachers perceived the T-TESS rubric to be a valid description of effective teaching (validity index ScaleM = 22.94, SD = 4.28). Teachers also believed that T-TESS evaluators had the ability to provide reliable ratings and coaching; however, nearly half of the participants did not have enough information to determine if rating reliability was equal between evaluators and across campuses (reliability index ScaleM = 14.36, SD = 2.98). Teachers perceived that T-TESS provided quality feedback that was useful for changing their instructional practices (feedback index ScaleM = 21.75, SD = 5.16). Survey responses indicated that some teachers misunderstood the general purpose for T-TESS, believing that T-TESS was used primarily for human resource decisions (purpose index ScaleM = 13.46, SD = 2.46). Taken all together, the independent variables of years of teaching experience, teaching a tested subject, number of observations, Julia M. Hyman, Ed.D.
Journal of K-12 Educational Research 13 amount of professional learning, and grade range were significant predictors of regression for the reliability index, feedback index, and purpose index. Table 1 provides a regression summary of the predictor variables. Qualitative analysis of participant responses supported the generally positive perception of T-TESS found in the quantitative data. Of the total respondents, 41% of teachers reported using T-TESS data to improve instructional practices, 10% to set and monitor goals, and 13% to earn a higher rating in successive evaluations. Those who reported using T-TESS data to improve instruction noted that the T-TESS rubrics, collaboration and feedback, and self-reflection were most helpful. Twenty percent of teachers reported no use for T-TESS data or no application of T-TESS to their specialized teaching situation, and 4% expressed concerns over fidelity of implementation and reported that participation in T-TESS was based on compliance and not beneficial to instructional design. Using the independent variables as a lens for qualitative analysis provided insight into the quantitative findings. Mid-career teachers were the most likely to use T-TESS data for instructional purpose. Special Education and elective teachers were the least likely to use T-TESS data as they perceived the data held little to no application in their specialized fields. Teachers who had experienced more observations were more focused on using T-TESS data for instruction while teachers with six or fewer observations were more likely to focus on using T-TESS to improve evaluation ratings. Professional learning had a positive influence on teachers using T-TESS to set goals and improve instruction. Elementary teachers were significantly more likely to report using T-TESS to improve instruction than secondary teachers. Secondary teachers reported a much more negative perception of T-TESS data use than elementary teachers. Forty-eight percent of all teachers reported a negative perception of including student growth measures on T-TESS, while 9% felt there was insufficient information to express an opinion on its inclusion. Sixteen percent expressed the perception that including student growth on T-TESS was logical as student performance should be the out-growth of effective teaching. A little over onefifth of teachers expressed reservations about the inclusion of student growth but leaned toward a positive perception. Those teachers with a negative perception or positive with reservations of student growth measures held similar concerns over fidelity of implementation, the challenges associated with measuring student growth, and fairness due to diverse classroom populations and outside variables over which teachers had little control. Late-career teachers, teachers not teaching tested subjects, and specialized teachers were most likely to express a negative perception on the inclusion of student growth measures on T-TESS. The number of observations and amount of professional learning had limited influence on the perception of including student growth. Secondary teachers were significantly more likely to report a negative perception of including student growth on T-TESS than elementary teachers. Implications for Research and Practice The findings from the current study extend the body of research in the field of teacher evaluation. The findings support the premise that teachers’ perceptions and beliefs influence their willingness to engage in teacher evaluations with the intent to change instructional practice. Additionally, research findings support the body of research on specific teacher groups for whom Table 1 provides a regression summary table of the predictor variables. Table 1 Summary of Multiple Regression Coefficients Note: * p ≤ .05; ** p ≤ .01; *** p ≤ .001 Qualitative analysis o parti ipant respons s supported the generally positive per eption of T-TESS found in the quantitative data. Of the total respondents, 41% of teachers reported using T-TESS data to improve instructional practices, 10% to set and monitor goals, and 13% to earn a higher rating in successive evaluations. Those who reported using T-TESS data to improve instr ction noted that the T-TESS rubrics, collaboration and feedback, and selfreflection were most helpful. Twenty percent of teachers reported no use for T-TESS data or no application of T-TESS to their specialized teaching situation, and 4% expressed concerns over Perception Index Predictor Validity Reliability Feedback Purpose Experience -0.037* -0.006 -0.009 -0.006 Tested -0.072 -0.082 -0.099 -0.077 Observations 0.008 0.007 0.004 0.010 Professional Learning -0.013 0.010 0.057 0.052 Grade Range -0.086** -0.173*** -0.175*** -0.135*** R2 .031 .060*** .071*** .066*** Table 1. Summary of Multiple Regression Coefficients Note: * p ≤ .05; ** p ≤ .01; *** p ≤ .001
14 instructional change through teacher evaluation based on general competency-based rubrics and student growth measures may prove to be a challenge: 1) late-career teachers, 2) secondary teachers, and 3) teachers of specialized programs such as special education and elective teachers. Educational leaders in Texas and in the District could use the findings from the current research study and past reform experiences to reflect on implications for practice. Recommendations for District Practice District leaders should clarify the purpose for T-TESS and clearly communicate the separation between the formative design of T-TESS and high-stakes personnel decisions. To reinforce this formative approach to teacher evaluation and address systemic function concerns expressed by teachers, District leaders should expand the cadre of evaluators beyond campus administrators to include District instructional leaders and teacher leaders (TEA, 2016). Use of these additional raters would alleviate issues with time and logistics, providing a knowledgeable group of evaluators who could serve as instructional mentors to teachers both in the core subjects and specialized programs. Deploying evaluators with specific content and program knowledge will improve observer capacity, data quality, and increase teacher perceptions of validity, reliability, feedback, and purpose. District leaders of specialized programs should develop supplemental documents to clarify descriptions of effective teaching practices for special education, advanced learners, and language learners to support application of the general competency-based T-TESS rubric. Additionally, professional learning and calibration of ratings for all evaluators of specialized programs would increase perceptions of rating reliability. As the District has identified student learning objectives (SLO) as the measurement of student growth on T-TESS, special effort to disseminate information on the use of SLO through district-wide professional learning is necessary. The District should develop exemplary samples of how teachers at each instructional level, content, or specialized program can establish, track, and use SLO data to improve instructional practices. District professional learning should include information on how student growth is to be defined and should account for variables outside of the teachers’ control in the measurement of student progress through SLO. District leaders should strengthen the capacity of the District to provide personalized, professional learning using T-TESS goalbased or rubric domain-based professional learning communities. Personalized learning could be expanded by establishing opportunities for teachers to observe peers identified as highly effective through T-TESS observations. Establishing expectations for teachers to expand use of T-TESS to include both goal-setting and improvement of instructional practice could establish a contextual setting for increased data use. To clarify confusion over the purpose of T-TESS, evaluators should be encouraged to engage teachers in dialogue using T-TESS data in coaching conversations, moving away from a check-list approach to conferences. Creating a strong connection between T-TESS data and professional learning may strengthen data use for its intended purpose and increase teachers’ positive perception of feedback. Recommendations for State Practice T-TESS has redefined the partnership between district and state leaders to support the design, implementation, and monitoring of teacher evaluation. In fulfilling this partnership, it is important for state leaders to collect and disseminate information on effective design and implementation efforts by districts across the state through the development of professional learning networks for district leaders. Providing resources and professional learning to 1) expand the cadre of teacher leader evaluators, 2) build understanding of connections between content and specialized programs and the T-TESS competency-based rubrics, and 3) increase skill in coaching conversations and data use would be possible through the state-wide Educational Service Centers. Development of exemplar case studies at each level of proficiency across instructional levels, within specific contents and specialized programs, and across student growth measures would provide concrete examples for study in districts that have implemented similar design elements for T-TESS. State leaders should continue to monitor outcomes of T-TESS and implement continuous improvement practices to refine state-wide evaluation processes. Julia M. Hyman, Ed.D.
Journal of K-12 Educational Research 15 Conclusion To achieve improved teaching quality and student performance through teacher evaluation necessitates that teachers believe there is a valid measure of effective teaching, ratings are reliable, feedback is useful, and there is a credible purpose for participation. The lesson from previous attempts to improve teaching quality through teacher evaluations in Texas is that consistent attention and outcome focus is necessary to realize the full potential of evaluation reform. Today, teachers stand at the crossroads between potentially positive outcomes or acceptance of nominal change in practice. District and state leaders have the ability to offer guidance down the more positive path by remaining committed to improving teaching quality by listening, heeding, and acting on advice from classroom teachers themselves. References Claudet, J. G. (1999). An interpretive analysis of educator change processes in response to a program innovation: Implications for personnel evaluation. Journal of Personnel Evaluation in Education, 13 (1), 47-69. Retrieved from https://link. springer.com/journal/11092 Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45 (6), 378-387. http://dx.doi.org/10.3102/0013189X16659442 Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: ASCD. Darling-Hammond, L. (1999). Teacher quality and student achievement: A review of state policy evidence. (R-99-1). Retrieved from University of Washington, Center for the Study of Teaching and Policy. Retrieved from https://depts. washington.edu/ctpmail/ Davis, A. (2013). Teachers' perspectives of improvement of instruction when engaging in self-assessment in a standards- based appraisal process (Doctoral dissertation). Retrieved from Proquest Dissertations and Theses Global. (UMI No. 3574587) Davis-Frost, D. (2000). Intentions and implementation of the professional development and appraisal system in Texas (Doctoral dissertation). Retrieved from Proquest Dissertations and Theses Global. (UMI No. 0478-0478) Eisenhart, M. A., Cuthbert, A. M., Shrum, J. L., & Harding, J. R. (1988). Teacher beliefs about their work activities: Policy implications. Theory into Practice, 27 (2), 137-144. http:// dx.doi.org/10.1080/004058448809543342 Ettema, E., Sengupta, K., & Kress, S. (2014). A legal lever for enhancing productivity. Retrieved from George W. Bush Institute website: http://bushcenter.imgix.net/legacy/ GWBI-Legal-Lever-for-Enhancing-Productivity.pdf Hampton, S. D. (2016). Teachers' perceptions of the Mississippi statewide teacher appraisal rubric (MSTAR) evaluation (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 10056548) http://aquila. usm.edu/cgi/viewcontent cgi?article=1339&context=dissertations Hargreaves, A. (2005). Educational change takes ages: Life, career and generational factors in teachers' emotional responses to education change. Teaching and Teacher Education, 21 (8), 967-983. http://dx.doi.org/10.1016/j. tate.2005.06.007 Heneman, H. G., & Milanowski, A. T. (2003). Continuing assessment of teacher reactions to a standards-based teacher evaluation system. Journal of Personnel Evaluation, 17 (2), 173-195. Herlihy, C., Karger, E., Pollard, C., Hill, H. C., Kraft, M. A., Williams, M., & Howard, S. (2014). State and local efforts to investigate the validity and reliability of scores from teacher evaluation systems. Teachers College Record, 116 (1), 1-28. Hill, H. C., & Grossman, P. (2013, Summer). Learning from teacher observations: Challenges and opportunities posed by new teacher evaluation systems. Harvard Educational Review, 83 (2), 371-384. doi:10.17763/ haer83.2.d11511403715u376 Hill, H. C., Charalambos, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41 (2), 56-64. doi:10.3102/0013189X12437203 Holmes, M. J. (2016). The impact of secondary teachers' perceptions, content areas, years of experience, and level of education on the Tennessee educator acceleration model Doctoral dissertation). Retrieved from Proquest Dissertations
RkJQdWJsaXNoZXIy ODc4ODgx