C. Standard 2.
The unit has an assessment system that collects and analyzes data on applicant qualifications, candidate and graduate performance, and unit operations to evaluate and improve the performance of candidates, the unit, and its programs.
C.1.How does the unit use its assessment system to improve the performance of candidates and
the unit and its programs?
The unit built and maintains a FileMaker database to collect and organize data relating to its BSE programs and candidates. A full-time database manager is a staff member in the unit. The database manager attends Program and Policies Committee (P & P) meetings and is an appointed member of the P & P Assessment Subcommittee.
The TEU collects data from candidates, faculty, university supervisors, and cooperating teachers. Candidates complete self-evaluations with regard to their knowledge and performance relative to the INTASC standards and with regard to their dispositions. They do the self-evaluation at admission to teacher education, when they apply to student teach, at mid-term of student teaching, and when they complete student teaching. Candidate scores on the PPST and the Praxis II and PLT tests are also stored in the unit database.
Faculty teaching courses in the professional sequence or methods courses submit scores on relevant INTASC standards at the end of each semester. A matrix showing which INTASC standards are addressed in each course is provided in the exhibits. Faculty input scores in the database, based on a four-point rubric. When assessing candidate knowledge or understanding of relevant ideas or processes, a score of 1, 2, 3, or 4 respectively is given to a candidate who shows little apparent understanding, somewhat na´ve or limited understanding, solid understanding, or sophisticated understanding. When assessing candidate performance, a 1, 2, 3, or 4 respectively is given to a candidate whose performance or product is ineffective, somewhat effective, effective, or highly effective. Candidate dispositions are also assessed using a four-point rubric, where a score of 1 warrants submission of the inappropriate dispositions form to TEAC, a score of 2 indicates that the candidate displays some inappropriate actions relative to the disposition, a score of 3 indicates that the candidate displays appropriate actions relative to the disposition, and a score of 4 indicates that the candidate displays sophisticated and consistent actions relative to the disposition. Faculty also submit the candidate's letter grade for the course. In addition to rubric scores and course grades, faculty supply a description of the assessments on which the submitted rubric scores are based and a description on how the rubric scores correlate to candidate performance on the assessments.
Candidates are assessed at five transition points during their programs of study. The first is at admission to teacher education. Candidates must meet all admission requirements in order to be admitted. The admission requirements include completion of the 3 general education communications courses (with a 2.50 minimum average and no grade lower than a C), minimum grade point averages of 2.50 in the major and for the cumulative GPA, completion of or enrollment in ED 250, attendance at Admission Seminar, documented 20 hours of activities with children/youth, completion of a speech and hearing screening, two satisfactory recommendations, inclusion of the candidate's current written statement of philosophy of education, the candidate's completed self assessment of dispositions and INTASC standards, and satisfactory basic skills as demonstrated by PPST scores, of which two out of the three scores must meet or exceed the higher number listed, the third score must not be less than the lower number listed, and the composite score must be 516 or more (Reading 173/170, Writing 173/171, Math 170/169.) The Teacher Education Administrative Council (TEAC) reviews all applications and determines whether or not an applicant will be admitted to teacher education. The registrar's office adds a formal statement indicating admission to teacher education to each successful applicant's transcript.
The second transition point is at application to student teaching. Each candidate must supply an autobiography and provide GPA information relative to the major and the professional sequence. They must also provide information on Praxis II test results if completed or provide information on when the Praxis II test will be taken. Each candidate must also submit a completed copy of the advising form obtained from the candidate's academic advisor and used to show progress in the program of study. The candidate's second self assessment relative to the INTASC standards and dispositions is also completed and submitted at this time.
The next assessment point is at the end of the methods course(s). Methods faculty submit data on most or all of the INTASC standards and on candidate dispositions. Faculty teaching other courses in the professional sequence (other than student teaching) do not submit data on dispositions.
The last two transition points occur during student teaching. The first occurs at mid-term and the second at the end of the student teaching experience. At both of these times, the candidate completes a selfevaluation relative to the INTASC standards and dispositions. The cooperating teacher and the university supervisor independently complete midterm and final evaluations of the candidate relative to the INTASC standards and dispositions.
Every fall the unit holds a retreat for its stakeholders. At the retreat program faculty and other stakeholders are given aggregated data from the unit database for all candidates in teacher education. The data provided includes demographics of candidates and students in P-12 schools, PPST data, Praxis II data, and data on INTASC standards and dispositions. Program faculty also receive aggregated data for the candidates in their programs at that time. The retreat offers an opportunity for representatives of the various stakeholder groups to interact and discuss issues relating to preparation of candidates. It is also a time when stakeholders can receive updates on program and policy revisions that have occurred during the past year. At the most recent retreat, a review of the conceptual framework was presented and Teacher Talks, a new initiative for cooperating teachers, was introduced. A teacher work sample model was also introduced and discussed as a possible way to improve data collection about candidate performance in the classroom.
The Assessment Subcommittee of P & P reviews the data annually. It looks at multiple years of data to determine if there are trends or if significant changes have occurred. The Assessment Subcommittee also examines the effectiveness of the unit's assessment system and determines whether data collection procedures are working and supplying appropriate, relevant, and usable data. It also addresses how the unit can assure reliability and fairness in its data collection and the alignment of assessment instruments with each other (graduate surveys, employer surveys, and instruments used to evaluate candidate knowledge, performance, and dispositions.) One of the items discussed by the committee this year was development of a plan to assess the candidate work samples provided during student teaching to show evidence of student learning. At this point, there is no data recorded based on the collection of the candidate samples. The question was then raised, why are we having candidates do this if we aren't using it for our data collection? There are valid reasons to include data from this collection of artifacts. We simply need to find a way to make it manageable and meaningful. Recommendations from the Assessment Subcommittee are taken to P & P, where further recommendations will be made to TEAC.
Surveys of graduates and employers of the unit's graduates are conducted on a regular basis, both at the initial and advanced levels. Survey data of BSE graduates and their employers are reviewed by TEAC, P & P and the Assessment Subcommittee. The Graduate School conducts annual surveys of its graduates and provides survey results to the Graduate Council. The MAT: Math program and the MEd Elementary/Middle School Mathematics Concentration are grant supported. Candidates in the programs are surveyed annually to collect data used in completing grant reports and in determining effectiveness of the programs.
Each advanced program has its own system for collecting and analyzing data relating to its candidates, program effectiveness, and curriculum. The Master of Education program and its 13 concentrations is establishing a database similar to the unit's database for the BSE programs. Transition points have been identified, and rubrics have been established for use in submitting data on candidate knowledge, performance, and dispositions.
All of the unit's programs are required to create annual assessment reports and submit them to the Vice President for Academic Affairs and to TEAC. The BSE programs must also complete a reflection/action plan on the assessment data regarding INTASC standards received at the annual stakeholders retreat and submit it to the Dean of the College of Education and Health Sciences.
C.2. Please respond to C.2a if this is the standard on which the unit is moving to the Target Level. If it is not the standard on which you are moving to the target level, respond to C.2b.
C.2a. Standard on which the unit is moving to the Target Level
- Describe work undertaken to move to the Target Level
- Discuss plans for continuing to improve
C.2b. Continuous Improvement
- Briefly summarize the most significant changes related Standard 2 that have led to continuous improvement. (If no significant changes related to this standard have occurred since the previous visit, indicate "None" in this section.)
Several program changes have been made as the result of analysis of data gathered at the program level. Those changes are described in the continuous improvement section in standard 1. Changes in the BSE Art curriculum were made in response to data received from surveys of exiting seniors, student comments and requests, and input from student teachers and their cooperating teachers. Changes in the BSE Mathematics curriculum and practicum were made as the result of data collected from a focus group of mathematics education majors on a university designated Assessment Day a few years ago. Some changes in the MAT: Mathematics program were made as the result of analysis of survey data collected each year as part of the supporting grant. Other changes, most notably the development of a capstone proposal course and capstone project, were made in response to the number of candidates who struggled to complete last part of the degree program, either a thesis or formal writing project of two papers. The new capstone project will also help candidates make a clearer connection between content knowledge, pedagogy, and research. The decision to drop the oral exam in the MAT: Math program came as a result of determining that little useful data about candidate knowledge or performance was being gleaned from the oral examination.
Since the BSE database now contains at least five years of data, although some of it is not consistent with current forms, the Assessment Subcommittee of the Programs and Policies Committee (P & P) has been able to create and analyze graphs showing trends.
The unit has made use of tools such as Google Forms to collect data on levels of preparedness of graduates from both graduates and employers. This technology has also been used to collect preferences of P & P members in the development and selection of language concerning candidate dispositions. An ad hoc subcommittee of P & P was created to formulate choices of language for the dispositions on each INTASC description. Five disposition wording options were developed for each standard. Those options were then sent to P & P members who ranked them 1 through 5. The technology used tabulated results and created graphs that were presented at the next P & P meeting for discussion.
The BSE unit database includes data on candidate GPAs as well as grades in required coursework and Praxis II scores. This data was used by the faculty in elementary education as the basis for their upcoming request to TEAC to change the elementary education requirements to include maintaining a minimum GPA in general education of 2.50.
Assistance was provided to the Accreditation Technology Support Coordinator to improve skills in the use of the unit database. Updates to the database software will allow us to expand reports to include graphs and charts directly from FileMaker. Our Accreditation Technology Support Coordinator also has developed the first video tutorial to assist faculty in providing accreditation data.
C.3. Exhibit Links
- Undergraduate - Description of the unit's assessment system in detail including the requirements and key assessments used at transition points [pdf]
- Graduate - Description of the unit's assessment system in detail including the requirements and key assessments used at transition points [pdf]
- Procedures for ensuring that key assessments of candidate performance and evaluations of unit operations are fair, accurate, consistent, and free of bias [pdf]
- Policies and procedures that ensure that data are regularly collected, compiled, aggregated, summarized, analyzed, and used to make improvements [pdf]
- Assessment Rubric for reporting INTASC data [pdf]
- INTASC reporting from undergraduate courses [pdf]
- Email reminding faculty of INTASC reporting responsibilities [pdf]
- Data from key assessments used at entry to initial programs [pdf]
- Graduate School Survey Data [pdf]
- Graduate School Survey Cover Letter [pdf]
- MAT Math Grant Report 2008-2009 [pdf]
- Graduate School Admissions Data [pdf]
- Policies for handling student complaints [pdf]
- P & P Assessment Subcommittee Minutes [pdf]
- All INTASC Standards with trend data [pdf]