Evaluating Teacher-Candidates ’ Teaching in the Extended Practicum

This study reports the aggregate results of the evaluations of the teaching performance of 17 cohorts of teacher-candidates (n = 374), who completed their extended practicum during their pre-service teacher-education program at one Western Canadian university. Over a 15-year period, one of the researchers served as the College Supervisor for these cohorts in their respective four-month internship in schools in a variety of locations throughout the province. The teacher-candidates’ final evaluations on nine categories of teaching performance revealed varying levels of strength among the skill areas. Implications of the findings are drawn: (a) for the undergraduate program administrators, faculty members, and practicum-program personnel at the university connected to this study, and (b) for other teacher-education institutions interested in making use of this information to help inform their deliberations in assessing/improving their own pre-service and practicum initiatives. Although there has been a call for teacher-education institutions to reform their pre-service programs in recent years (Council of Ministers of Education Canada, 2002; Wideen & Lemma, 1999; Yackulic & Noonan, 2001), the overarching mandate of teacher education has not changed. That overall goal has always been, and will continue to be, to prepare reflective, self-evaluatory practitioners capable of developing and utilizing a repertoire of professional knowledge and skills to promote student learning (Anderson & Burns, 1989; Richardson, 2001; Wittrock, 1986). An essential component of this preparation process common to most programs has been the school-based extended-practicum experience. In this extended practicum (sometimes called the internship) teacher-interns (i.e., the student teachers or practicum students enrolled in the extended practicum) Brock Education Vol 14, No. 1, 2004 1 Edwin Ralph is Professor and Graduate Program Coordinator in the Department of Curriculum Studies. His research interests lie in the areas of instructional development (and its supervision) at both the K-12 and the post-secondary levels. Brian Noonan is Associate Professor and Graduate Program Coordinator in the Department of Educational Psychology and Special Education. His research interests lie in the areas of research methodologies and applied measurement and evaluation. E.G. Ralph, B.W. Noonan 2 —under the guidance of their site-based cooperating teachers and their college of education advisors—develop their teaching confidence and competence, as they seek to internalize a variety of effective pedagogical and instructional practices (Beck & Kosnik, 2002; Bullough et al., 2002). Important questions that logically arise about this practicum component of pre-service teacher education are: How well do these extended-practicum candidates perform in their teaching practice at the conclusion of the internship? What instructional skills are their strongest? Which ones are weakest? and What implications would such findings have for the teacher educators responsible for the program? These questions formed the basis of the present research inquiry, as we in our College of Education sought to examine the effectiveness of our teacher-education efforts. Our search of the literature revealed a serious gap in research devoted to such questions about teacher-interns, which provided us with a further incentive to pursue such an investigation.

-under the guidance of their site-based cooperating teachers and their college of education advisors-develop their teaching confidence and competence, as they seek to internalize a variety of effective pedagogical and instructional practices (Beck & Kosnik, 2002;Bullough et al., 2002).
Important questions that logically arise about this practicum component of pre-service teacher education are: How well do these extended-practicum candidates perform in their teaching practice at the conclusion of the internship? What instructional skills are their strongest? Which ones are weakest? and What implications would such findings have for the teacher educators responsible for the program? These questions formed the basis of the present research inquiry, as we in our College of Education sought to examine the effectiveness of our teacher-education efforts. Our search of the literature revealed a serious gap in research devoted to such questions about teacher-interns, which provided us with a further incentive to pursue such an investigation.

Background
We, the researchers who conducted this study, agree with Wilen, Ishler, Hutchison, and Kindsvatter (2000, pp. xi), who stated: "A sound knowledge base is the bedrock of every bona fide profession. Translating this knowledge base into thoughtful classroom practice is the challenge and task of every professional teacher." This body of professional knowledge and skills is considered necessary, but not sufficient, to achieving teaching effectiveness (Darling-Hammond, 2001;Good, 1990;Palmer, 1993). Moreover, the considerable body of research literature that has accumulated over the past four decades has consistently demonstrated that teachers who possess and apply this knowledge in their practice are more effective than those who do not (Anderson & Burns, 1989;Rosenshine & Stevens, 1986;Stronge, 2002).
At our College of Education, we have synthesized this research literature into a 200-page document called The Internship Manual (University of Saskatchewan, 2004Saskatchewan, -2005 that classifies the professional-knowledge base into nine essential teaching categories. This manual is used by all interns, their classroom cooperating teachers, and college supervisors during the extended practicum to guide their respective teaching and supervisory activities throughout the 16-week session. The session comes in the fourth and final year of teacher-candidates' B.Ed. program. The college supervisor works with cohorts of 20 to 25 intern/cooperating pairs in a jurisdiction, and conducts monthly full-day seminars examining selected instructional methods, strategies, and skills with the cohort. The college supervisor also visits each pair in their respective schools/classrooms each month, and facilitates the classroomteacher's mentorship of the intern as they develop their teaching skills. The nine instructional categories in The Internship Manual serve as the basis of: (a) the interns' teaching activities, (b) the supervisors' mentoring process, and (c) the evaluation criteria on which the interns are assessed (formatively, during daily and monthly supervisory conferences and at the formal mid-term evaluation; and summatively, at the collaboratively conducted final evaluation at the end of the practicum).
A summary of the nine teaching areas is as follows: Personal and professional attributes. Teacher-interns are expected to demonstrate commitment to establishing and maintaining positive relationships and ethical conduct in all of their activities. There are 13 sub-skills that candidates are to internalize, and upon which they are evaluated, two of which are: • Show initiative and leadership by involvement in school activities. • Exhibit affective attributes such as warmth, patience, tolerance, empathy, and respect. Lesson planning. Teacher-candidates are expected to compose daily, written lesson plans, and to prepare and organize the necessary materials. There are 10 sub-skills, two of which are: • Include a motivational set for creating student interest. • Plan an effective sequence and time-frame. Unit planning. Interns are to plan a minimum of two formal units of work incorporating eight specific criteria, two of which are: • Define major concepts and key objectives.
• Integrate related content with other subject areas. Presenting. They are to demonstrate 12 sub-skills in effectively "delivering" the planned teaching and learning activities. Examples of two of these presenting sub-skills are: • Give clear directions to pupils.
• Provide pupils with guided practice and feedback/correctives, as needed (and reteach if necessary).
Classroom management. Teacher-candidates should be able to incorporate the 13 strategies/skills found to be effective in creating the conditions in the classroom situation for teaching and learning to take place. Two examples are: • Gain attention of all pupils before beginning the lesson and at transition times. • Demonstrate awareness of inappropriate pupil behavior and implement effective interventions. Questioning. Interns are expected to internalize five key questioning skills that will enhance pupil learning, two of which are: • Demonstrate clear, concise questioning patterns (by avoiding six common errors). • Distribute questions equitably among pupils. Responding. They are to demonstrate eight skills that encourages pupil learning and builds pupil feelings of selfworth. Two examples of teacher responding behavior are: • Avoid repeating ("echoing" or "parroting") pupil answers. • Reinforce correct portions of pupil answer and probe effectively for more. Employing a variety of instructional methods. Teachercandidates are expected to develop and utilize a repertoire of teaching methodologies/strategies in order to meet particular instructional goals and/or learning needs of pupils. Using the 18 methods described in The Internship Manual, interns are to demonstrate five skills, two of which are: • Enhance pupil acquisition of concepts, abstractions, and patterns of relationships. • Enhance pupils' creative thought and to deal with experience of reality. Assessment/Evaluation of pupils. Interns are expected to develop seven sub-skills in this area, two examples of which are: • Use a variety of formative and summative assessment techniques. • Maintain records of formative and summative assessments for reporting purposes.
These nine categories are the source of: (a) the objectives for the practicum, (b) the knowledge-and skill-set that the interns develop throughout the term, and (c) the criteria upon which the candidates' teaching performance during the internship is judged. All of this material and the related supervisory and evaluatory procedures are made explicit to all stakeholders from the beginning of the practicum program through to its conclusion.

Participants
We examined the final evaluation forms of 17 cohorts of teacher-interns, for whom one of the researchers had served as internship college supervisor for their extended-practicum. This sample (McMillan & Schumacher, 2001) of teacher-candidates (n=374) was drawn from the population of all interns who completed their extended practicum from 1987 to 2002 through our College of Education (approximately 6000 pre-service teachers). The sample consisted of one cohort of approximately 25 interns per year over the 15-year period. Each of these cohorts was representative of the annual number of approximately 400 teacher-interns, in terms of age, gender, subject-major/-minor, grade and school placement, and urban/rural location. The college supervisor had no influence on the composition of any of his cohorts; and his assignment to, and the distribution of, the 15 cohorts was done randomly by the field-experience office administrators.

Instrument
The instruments used were the evaluation forms completed during the final three-way post-conference with the intern, cooperating teacher, and college supervisor. On this form a collaborative decision was recorded regarding the then-current performance of the intern with respect to the nine teaching categories on which the internship was based.

Procedure
Using the SPSS 12.0 program (SPSS, 2003), we calculated the means and standard deviations of the numerical equivalencies of the evaluation ratings of the nine teaching skills that were recorded on the final evaluation forms for each of the 374 teacher-interns. These values were converted from the prescribed set of descriptive ratings as described in Table 1.

Evaluation-Description Numerical Value
Strong 3 Demonstrated the sub-skills in this category consistently, effectively, and independently. Will have no difficulty in this area during the first year of teaching.
Competent 2 Demonstrated some of the sub-skills in this category, with consistency, effectiveness, and independence. Will require some assistance in order to be successful in this area during the first year of teaching.

Concern
1 Experienced difficulty in demonstrating consistent, effective, and independent performance of the sub-skills in this category. Will experience difficulty in this area during first year of teaching.
Failure* 0 Performance in this category was clearly unacceptable.
Note. No intern in this sample received a failure rating on their final evaluation, because an intern experiencing such serious difficulties (after deliberate supervisory attempts to assist them) would have withdrawn (or have been withdrawn) before the completion of the final evaluation.
There were no "0" values recorded because any intern who had experienced serious difficulties in the practicum had previously withdrawn from the program prior to the completion of the final evaluation. This action may partially account for the relatively high ratings of the candidates' performance, because the lowest rankings were thus not included in the calculations. The College's statistics show that approximately 4% to 5% of the interns placed each year in the province's schools withdraw for reasons of inadequate performance.
The descriptive statistics for the teacher-interns final evaluations in the nine instructional categories are shown in Table 2. Note. The scale of values for the performance evaluations, as shown in Table 1, included 3 for "Strong", 2 for "Competent", and 1 for "Concern". "0" was not used.

General Findings
An examination of the values shown in Table 2 reveals three general findings about the interns' teaching performance. One finding was that although all of the means were in the "Competent" category (i.e., with a value within the 2 category), they were distributed between 2.82 and 2.33 indicating varying degrees of strength of performance in the skills for the sample. A second general finding was that the categories with the three highest means (i.e., lesson planning, personal/professional attributes, and unit planning) were not observable components of the "live instructional act" that occurs in the actual interaction of a face-to-face lesson. Instead they were associated with important requisite elements to the active delivery of the teaching/learning event, that is to the planning and/or the psycho-socioemotional components of the teaching/learning process. A third related observation was that although the six means connected more directly to the teaching act (i.e., items #4 through #8) ranked lower than the planning and psycho-social aspects, they were relatively close in value to one another, ranging between 2.38 and 2.47.

Specific Results and Discussion
In this section we analyze the evaluation ranking of each of the teaching categories and discuss selected implications of these data for our College's program administrators as they seek to improve our teacher-preparation initiatives. Although such specific findings may not be generalizable to the programs offered by other teacher-education institutions, the leaders of these programs may be able to gain from our findings some information having "comparability," "translatability," "fit" (Ward Schofield, 1990, pp. 208, 226), or "transferability" (Donmoyer, 1990, pp. 182-190) to help them better analyze and understand their own unique situations.

Instructional Planning
The values in Table 2 indicate that the teacher-candidates' planning was highly rated, with Lesson Planning being notably stronger than Unit Planning. The research literature (Clark & Peterson, 1986) has consistently shown that teachereducation institutions typically emphasize lesson planning for teacher candidates, and that the latter become proficient at it only when they are required to do so. Practicing teachers, on the other hand, favour longer-range unit planning, centered around the general flow of activities over the unit's timeperiod, rather than the fine details of each lesson (Sardo, 1982).
An implication of these findings for program-planners and instructors in our pre-internship methods courses is that we need to provide the teachercandidates with more prior exposure to planning instructional units, with specific reference to the format presented in The Internship Manual. This goal, in fact, is currently being pursued, because at the time of the writing of this report, one of the authors had just been assigned as instructional-coordinator for the mandatory core-course taken by all teacher-candidates in the college. This course, an introduction to the fundamental concepts of curriculum and generic teaching skills, is the preparation for candidates' subsequent methods courses, their student-teaching experiences, and the four-month extended practicum.
By having an instructional-coordinator for this pre-practicum core course, we will seek to ensure that all instructors teaching its sections will emphasize with all pre-interns the unit planning components that, to this point, may not have been as evenly and as systematically presented as they could have been. We propose to have regular meetings with all instructors of this course, in order to highlight the areas requiring attention, and to share among the instructional team the strategies and materials that will bolster the strengths and ameliorate the weaknesses that have been identified in interns' planning and teaching.

Personal and Professional Attributes
Teacher-candidates appear to have a sound grasp of the human element of the teaching enterprise, and of their personal and ethical responsibilities to the various stakeholders involved in the schooling process. This finding would resonate well with the province's school division administrators, who indicated in a previous study about their hiring preferences for new teachers (Ralph, Kesten, Lang, & Smith 1998), that the most important criterion they hold is that prospective hires "have demonstrable strengths in interpersonal communication and classroom management" (p. 53).
Possible reasons for teacher-interns' high performance in this category are: (a) the emphasis that the College places throughout all of its courses and practicum experiences on professional and ethical conduct of students, instructors, and supervisors; and (b) the caliber of students who apply and are accepted into the College each year reflect this emphasis.
The College's course instructors and practicum personnel will need to continue their commendable work in supporting this quality, in order to help the upcoming teacher-candidates to embrace and maintain this professional stance for which their predecessors have been noted.

Presenting
Among the six remaining categories shown in Table 2, which in fact relate to the "live" teaching behaviours that are delivered in active classroom performance, the interns' presenting skills are ranked highest. Teacher-interns and their mentors alike should be pleased-but not complacent-with this finding, because the research literature has long indicated that these presenting/ structuring skills form the heart of teaching any structured portion of any content area (Borich, 2003;Kounin, 1970;Rosenshine & Stevens, 1986).
In the core methods course mentioned above that one of us will be coordinating, we will ensure that all instructors of the course describe, demonstrate, and have participants perform (and conduct collaborative peerand self-evaluations of) videotaped microteaching episodes. In these sessions they practice the presenting skills, and give and receive constructive feedback -including that from the instructor (and that from their supervisors during student teaching and the extended practicum). This clinical, professionaldevelopment process has been shown to be consistently effective in helping practitioners internalize new skills (Hargreaves, Earl, Moore, & Manning, 2001;Showers, Joyce, & Bennett, 1987).

Classroom Management
Classroom management can be described as creating and maintaining the conditions conducive to the teaching/learning process (University of Saskatchewan, 2004Saskatchewan, -2005, and it has been long regarded as a crucial component of effective teaching by novices entering the field (Ralph, 1993(Ralph, , 1994Veenman, 1984); by veteran teachers (Brown, 2004;Powers, 1992); by school division administrators (Jensen, 1986;Ralph et al., 1998); and by educational researchers (Doyle, 1986;Jones & Jones, 2004).
Our present study shows that although beginning teachers were competent in this area of instruction, their ranking at a point midway among the nine categories suggests that there is room for improvement. The research literature on beginning teachers' competence in classroom management indicates that it is not only an ongoing concern to them, but that if these novices do not master the required skills in organizing and managing classroom learning, then problems inevitably escalate, leading to even more serious consequences (Fullan, 2001;King, Warren, & Peart,1988). A specific response to this finding that our College is currently initiating is to move the mandatory classroom management course from being taught concurrently with the extended practicum to being taught prior to it, in conjunction with a two-week student-teaching session. By enacting this change, we will seek to accomplish three goals: (a) to reduce the level of frustration expressed by past interns who reported the extra load of taking another course-in addition to performing all of the duties required of the internship-to be impractical and overwhelming, especially when it was taught by instructors who were not connected to the practicum; (b) to connect the material with actual classroom life, but to do so prior to the extended practicum and within a shorter time-frame; and (c) to provide candidates with specific prior knowledge upon which they could build strategies when they begin the later internship.
We anticipate that this change in format will help raise the effectiveness level of classroom management performance among our future teacher-interns.

Responding
Because teachers' responding behaviours (described as what the teacher says and does after students' comments or actions, University of Saskatchewan, 2004Saskatchewan, -2005 are so critical to the quality of the psycho-socio-emotional atmosphere that pervades the classroom and school environment, these behaviors are considered essential for all teacher-candidates to acquire and use (Borich, 2003;Jones & Jones, 2004;Good & Brophy, 2002). Neophyte teachers must internalize a variety of relevant communication skills to better prepare them to deal effectively with increasingly complex challenges in contemporary schools (Emmer, Evertson, & Worsham, 2002;Levin & Nolan, 2003). However, the value found for responding shown in Table 2 suggests that our interns will need to improve their skill in this area.
Some of the challenges that interns face in schools include: accommodating multicultural diversity, working with children and youth who are at-risk, facilitating the inclusion of exceptional children, dealing with ethnic/ racial/social class/language/gender differences, mediating interpersonal conflicts and confrontations (e.g., bullying and violence), and communicating with parents and a variety of community agencies (McCown et al., 1999;Woolfolk, Winne, & Perry, 2002). Moreover, previous research assessing our own pre-service program revealed that our College was not doing a particularly effective job of preparing our interns to deal successfully with these issues (Ralph,1995). Yet other research has shown that teachers' verbal and non-verbal responses in both the learning/teaching situation and in the non-academic milieu of school life do exert a powerful influence on pupil motivation, attitudes, and feelings of self-worth (Wilen et al., 2000). Therefore, within the core methods course mentioned earlier, we plan to have all instructors require our teacher-candidates to practice some of these skills during their microteaching sessions.

Assessing Pupil Learning
This present study has also confirmed the results of previous research on our internship program that identified interns' perceptions of their lack of knowledge in this area (Ralph, 1998(Ralph, , 2002(Ralph, , 2004. Thus, an inference raised in one of those previous papers is pertinent here and is worthy of re-emphasis: "An implication of this finding for the practicum organizers and supervisors in our program is that we need to provide more learning experiences for pairs [interns and their classroom cooperating teachers] to develop their competencies in student evaluation" (Ralph, 1998, p. 25). The relatively low rating for this category as revealed in Table 2 attests to this gap in the interns' teaching performance.
Fortunately, our College has used these findings as a basis for considering to enact the following changes to our agenda of methods coursework: (a) a compulsory course on assessment/evaluation, and (b) requiring instructors in all of our methods courses to ensure that they include an evaluation component in their courses. In the meantime, we will require the instructors in the core methods-course that we are coordinating to include a basic treatment of the key evaluation concepts and techniques, so that all teacher-candidates will have a fundamental grasp of the skills before they begin the practicum.

Questioning
The relatively low rating for teacher-candidates' oral-questioning skills also confirms earlier research from two sources: (a) American findings that teachers, in general, are less effective in their questioning than they could be (Anderson & Burns, 1989;Dantonio, 1990); and (b) our own findings that show a relatively low ranking of effective questioning by interns during the extended practicum (Ralph, 1995). Teachers, whether novices or experienced, typically do not incorporate the skills that research has shown to enhance their instructional questioning (Crowl, Kaminsky, & Podell, 1997;Goodlad, 1984). A reason for this deficiency may be that to internalize and to incorporate such techniques takes considerable effort and sustained practice. Showers et al. (1987) found, for instance, "that about 25 teaching episodes during which the new strategy is used are necessary before all the conditions of transfer are achieved" (p. 86). Hence, many veteran teachers-let alone the teacher-candidates during their pre-service programs-may not devote the time, or have the inclination, to engage in this intensive process. However, what is even more significant, is our previous research that has shown that teacher-interns can in fact improve their questioning skills during the practicum by means of a sustained period of deliberate practice in actual classrooms, provided it is accompanied by specific feedback and support from their classroom cooperating teachers and college mentors (Ralph, 1999a(Ralph, , 1999b. In the light of these findings, it is our intention to help interns in our program to increase their questioning competency by having the core-course instructors whom we will be mentoring devote a portion of their coursework to good questioning, and to have pre-interns incorporate clear oral questions into their microteaching episodes (and thereby include the requisite peer-and selfevaluations that are a part of the microteaching). We will also have these instructors emphasize the vital connection that exists between effective questioning and classroom management (Good & Brophy, 2002;Hunkins, 1989;Rowe, 1986). Thus, by incorporating all of these elements in our pre-service program we believe that the evaluation-ratings for our teacher-interns in the questioning category will improve, thereby enhancing their overall teaching performance.

Variety of Instructional Methods
The finding that interns' use of a variety of teaching methods was an area requiring improvement had also been identified in previous research regarding our extended practicum (Ralph, 2002(Ralph, , 2004. In The Internship Manual's section on teaching methods 18 of them are described (e.g., brainstorming, computerassisted instruction, concept attainment, cooperative learning, discussion, demonstration, inquiry, peer tutoring, talking circles). Interns are encouraged to implement as many of these methods as possible throughout the 16 weeks, but this goal has not been completely achieved, as suggested by the 2.33 mean shown in Table 2.
Possible explanations for this low rating are that: (a) although the practicum organizers and the interns "know about" these methods, the latter are not familiar enough with them to apply them effectively in the day-to-day rigors of classroom routines; (b) many classroom cooperating teachers may not themselves be comfortable enough with all of the 18 methodologies to coach the interns appropriately in the implementation of them; (c) the instructors in the campus-based methods courses may not be adequately preparing the interns to utilize these methods prior to the internship; or (d) the internship mentoring personnel may not be applying Contextual Supervision, in which the mentors carefully provide enough task-direction and supportive encouragement to bolster interns' lagging levels of competence and confidence in this area (Ralph, 1993(Ralph, , 2003.
Research on the success of Contextual Supervision (CS) model shows that interns' teaching performance can be enhanced when supervisory personnel correctly match their mentorship styles to meet the existing developmental needs of their protégés, and then adjust these styles according to the changing levels of their interns' performance (Ralph, 1996(Ralph, , 1998(Ralph, , 2002. We are certain that the same results could occur for the category of applying instructional methodologies, if we were to prepare the annual group of 12 -15 college supervisors to implement the CS model with their respective cohorts of 20 -25 practicum pairs.

Concluding Thoughts
This study has indicated that our teacher-candidates, as a whole, are performing competently in all of their instructional skills. Instructional planning and professional conduct were areas of strength, but the actual "face-to-face" skills related to live classroom interaction could be improved (i.e., presenting, classroom management, questioning, responding, evaluating students, and incorporating multiple methodologies). Implications we have identified for teacher educators in our college relate to specific amendments to strategic components of the entire pre-service program. We believe that these revised strategies show promise for enhancing our candidates' pre-service teaching performance. Some of these changes are: coordinating instructors' teaching of our core-methods course; moving the compulsory classroom management course to a more suitable time-frame; having pre-interns practice their instructional skills (and have them provide and receive constructive feedback with their peers) in microteaching sessions; and incorporate the Contextual Supervision model with all cohorts of internship pairs to help them internalize a variety of teaching methods.
However, we would be naïve to think that our future interns' final evaluation-ratings would improve by the mere incorporation of such changes into our existing program. Rather, we agree with Hargreaves et al. (2001, p.131-132, from their study of how teachers learned to implement new curricular/ instructional initiatives): "This kind of change does not happen by osmosis, administrative mandate, or even sheer will and determination" but rather, they found that such improvement occurred when teachers' "outside-in knowledge… was filtered through inside-out experience." In other words, we believe that when candidates are convinced of the relevance/benefits of these skills for improving their own teaching success and the learning and interest of their pupils, then their motivation to embrace these techniques will increase as a result (Stronge, 2002). It is also our desire that our colleagues in other teachereducation institutions will be able to extract insights from their reflection on this present study in order to help inform their deliberations to enhance their own programs.