(Provided by the Contract Administration Committee)
1. How effective is it going to be to deploy the SIR II survey to students only through their email addresses?
Concerns regarding accuracy of email addresses –
We understand that no matter what we try to do, a student’s email address may not be accurate or real. It could have been entered incorrectly at the time of registration, unused by the student, or the HP 3000 email address may be different than the Blackboard email address. When we embarked on this project, we thought that all students would be assigned an EdCC email address. Plans for that have been delayed but are still moving. Right now a computer program has been written that updates student email addresses. This program changes the default email to the current email a student has in Blackboard. In spite of this technology fix, there have been numerous reports of students not receiving the evaluation email from ETS. We are working with ETS on this now. Instructors can help by reconciling their students email address early in the quarter –weeks 1 and 2- (making sure students got the email they were sent via Blackboard). The good news is that only 686 emails were returned to ETS from over 12,000 sent to students.
Concerns that only dissatisfied students might be motivated to complete the surveys to vent
There is currently no evidence that this might happen nor is there evidence that the opposite might happen either. The computer program that screens student email addresses also take students out of the download who have withdrawn. We will be checking with our SIR II contacts to see if there is any research on this or not and also to contact colleges who use web based student evaluations to see if there is any more information on the topic.
Concerns that some students might not get the notification
Students are being notified in multiple ways. One is an email or Bb announcement from their instructor on the SIR II, ACS labs will have a banner announcing the survey period and announcing the importance of completing the survey. Some divisions are sending letters to students in classes and posting signs in classrooms notifying students of the evaluation opportunity. We encourage instructors participate in reminding students to participate, too. The major concern right now is that some classes did not get in the download and have been redeployed, and there are other reports by students who have not received the ETS email. We know some student email boxes were full and those were returned, some others had gone to junk mailboxes, and some we don’t have an answer for yet. This why were are piloting this survey to see where the bugs are.
Concerns that students in grounded classes that do not or use only rarely Bb might not participate
This is a major change in culture for our college. It is early in the pilot and we just don’t know yet. We are contacting other colleges that only use web-based surveys to see if this is a consequence. We are also going to look at our own data and see if this is a reality.
2. How will the survey results “count” in a faculty member’s personnel file?
Student evaluation of faculty is only one part of the performance appraisal process. We use, based on accreditation standards, multiple indices. Usually there are several years of student evaluations in the working folder. These pilot surveys would just become part of that file.
3. How will they be used by administration?
Since performance appraisal is based on a number indicators, this is only one part. We plan to take the SIR II suggestion and look at multiple classes of evaluations or these new SIR II forms in comparison with previous student evaluations. We also have the opportunity to look at national data in specific subject areas. One of the great benefits of the SIR II is the Compendium: “Enhancing Your Teaching Using the SIR II” which is a great resource for faculty and administration alike to use to improve teaching and learning. Already the faculty development coordinator is looking for faculty who are skilled in specific sections of the SIR II to act as mentors to faculty who wish to make improvements.
4. What’s the standard we are expected to meet? What does admin want?
No numerical standard has yet been set. We realize this is a major cultural shift and the move away from a “cut off score” leaves uncertainty. One rule of thumb to self assess your own effectiveness would be to look at your individual class report and sum the percentages of the responses that came in effective or very effective by item or by section. Another way might be to compare your class report with the “Two year Colleges and Technical Institutions Percentile Distributions of Class Means” in your subject area, for part-time faculty, full time faculty, classes conducted as lecture and discussion combined, or classes conducted as laboratories. There is a lot of data available and when we have enough data from the pilot, we may establish some benchmarks.