Departmental Student Learning Project at Cal Poly Pomona
Dr. Nancy Page Fernandez, Chair (2000-2008)
Interdisciplinary G.E. Department
Cal Poly Pomona
Using Student Evaluations of Teaching to Improve Student Writing:
The Back Story
Student evaluations of teaching—those #$%@
bubble forms—are used more than perhaps any other measure to obtain
feedback about student learning at the university. Your campus policies
may even require departments to regularly use this summative feedback
to inform personnel decisions. And this practice persists in light of
studies that point to significant limitations to focusing primarily on
students’ evaluations of faculty teaching ability.
While bubble forms are easy to score, they are difficult to interpret.
The standardized questions often ask for judgments about teaching skills
and methods (which students are not qualified to evaluate) instead of
asking for feedback on instructor contributions to student learning (something
they may be able to evaluate, but only for the short term). Standardized
evaluations are often perceived as popularity contests or teaching competitions,
which may work against fostering an environment where students and instructors
work as partners to improve learning and teaching. In the aggregate even
valid and reliable questions provide little information about how to
improve student learning. How can we go beyond the bubble to gain meaningful
feedback from students about their learning experiences?
The IGE Department at California State Polytechnic University, Pomona,
uses several different student assessments to evaluate teaching and learning.
collective dissatisfaction and frustration with the limits of the bubble
form instrument led us to revise student evaluation of teaching questions
to align with our learning outcomes.
We initially hoped to gain more specific feedback on student learning experiences. It remains unclear if that objective was met. In the end, however, we learned a lot through the process.
We began our project in 2003-04. Over many meetings and discussions we:
- Reviewed the current form to discuss which learning outcomes were reflected in the questions, and which outcomes were absent.
- Collected and analyzed survey forms from other departments to learn their approaches.
- Drafted language for new questions.
- Consulted with colleagues with expertise in survey research for their critiques of our draft questions.
- Narrowed to 15 total questions.
- Implemented the new form in spring 2004.
One question surprised us:
No. 13: “Instructor provided meaningful feedback on student writing.”
The first quarter we administered the new forms, question 13 received
the poorest overall average score and the largest standard deviation. We
learned that providing meaningful feedback on student writing was a struggle
that we all shared, hence presenting an opportunity for us to work together
as a department to improve student learning and program effectiveness.
We embarked on additional assessment and faculty development in the area of teaching written communication, whereby
- We examined pre- and post-course self-evaluations to explore students'
perceptions of writing strengths, needs and problems.
- Instructors conducted mid-point assessments to gain student feedback on their learning experience.
- Students' introductions to their course portfolios provided additional reflections and self-evaluations of their learning.
- In our regular exit interviews with students, we included questions about their experiences learning to develop their writing abilities.
- Faculty with expertise in composition facilitated a workshop for department colleagues.
- The department purchased new and respected books on composition pedagogy
and circulated recent journal articles on developments in writing instruction.
- Individual instructors participated in intensive workshops offered by the Faculty Center.
- Individual instructors attended Brown Bag Workshops offered by the Faculty Center.
- Faculty members shared their experiences and ideas at department meetings and the annual retreat.
Getting Surprised (again)
Through discussing how we support students' development of written communication skills, we learned that instructors had different expectations for student writing. We decided to articulate expectations for student writing performance in a "Written Communication and Critical Thinking" rubric.
Getting Working (again)
- We shared the wide variety of types of writing we use in our classes
and how we understand the relationships between writing and thinking.
- We explored the roles writing plays in developing critical, creative and original thinking.
- We reflected on what we had learned from multiple sources of assessment data about the students' formative learning.
- Veteran instructors searched their files to locate the original IGE Written Communication and Critical Thinking rubric.
- We collected sample rubrics from English departments and interdisciplinary programs.
- We piloted a variety of rubrics in our classes and collected student feedback on their user experiences.
- At our June 2007 retreat we successfully drafted a new rubric!
- We implemented the rubric in fall 2007.
During the 2008-09 academic year, we will use data from student self-evaluations, exit interviews, focus groups, and reviews of student portfolios to assess the effectiveness of the new rubric. We are continuing our discussions of student writing, and we are establishing expectations for formative development of written communication skills over the sequence of IGE courses.
What We Learned
- Effective assessment of teaching requires a cohesive program that integrates data from a variety of sources (not just the #$%@ bubble forms).
- Approaching assessment of teaching as part of program evaluation can enhance opportunities for peer mentoring and faculty development.
- Assessment of teaching is really about assessment of student learning, which is an important part of being a learning-centered department.
California-Chile Agreement Takes Off
Jeff Gold, Cynthia Desrochers, and
Juan Manuel Molina, President (and flight instructor)
Apoquindo University, INACAP
September 7, 2008, Santiago, Chile
As part of the California-Chile agreement made by California governor Arnold Schwarzenegger and Chilean President Michelle Bachelet, Jeff Gold (Academic Technology) and Cynthia Desrochers (Institute for Teaching & Learning) traveled to Chile to consult with INACAP on their master's of Pedagogy in Higher Education degree. INACAP, the 25-campus Universidad Tecnologica de Chile, chose to collaborate with the CSU because of similarities between the two systems in mission, challenges, and goals. Moreover, California and Chile share broadly similar climatic conditions along the Pacific Ocean. In addition to opportunities for CSU faculty to teach in the INACAP master's program, this agreement will also create opportunities to collaborate in applied research on energy, water, and human capital.
||Teaching & Learning Tips
Freshman Focus at Cal State Northridge
Motivating Metamorphosis: Teaching College Freshmen
There's no way not to notice how much thought and effort American corporations put into the task of getting teens to act as consumers and spenders. Commercials in magazines, on television, at the movies, on billboards, on t-shirts (and other imprintable garments), and of course online take direct aim at this age group. They use loud music, bright colors, flashing lights, compelling rhythms, memorable tag lines, and the lure of sex and sexuality to entice teens to buy, wear, drink, eat, drive, and love their products and services. But what about us? In our classrooms, how do we get these teens (whom we know as "freshmen" or "first-year students") to invest their time and attention in us? How can we motivate them to read our messages, consume our goods, and buy our lines in the classroom? Put bluntly: how can any of us hope to compete with the ad campaigns of Abercrombie and Victoria for the minds of our freshmen?
My own starting point is a dogged determination never to give up: I'm in this field for keeps, for good. I suspect you are, too. Here are some other things I think we can do.
Dr. Cheryl Spector
Director of Academic First Year Experiences
1. Build on the students' social nature.
They are keenly aware of one another at all times and in all arenas, including our classrooms. Therefore, why make even your briefest quizzes moments of private anguish when you could instead let students vote their answers to each quiz question publicly? For instance:
Sound like trouble? Consider that you'll get instant feedback on what your students are learning, without the need to grade every quiz outside of class. (You can still administer traditional quizzes, but use "public quizzes" as a variation or an addition to your repertoire.)
- by a show of hands, or
- by displaying brightly colored 3 x 5 cards: fluorescent green for YES (or A), hot pink for NO (or B), etc.; or use school colors, or
- by physically moving to one side of the classroom, to indicate their level of agreement or disagreement with a statement, or
- with clickers.
2. Build a scaffold for your students.
Lead them step by step through (or to) college-level writing, reading, thinking, and/or speaking. Help them break down large tasks into smaller and more manageable pieces by including official due dates for each piece on your syllabus. A suggestion from my colleague Mary Riggs: give them forms that prompt them, prod them, and structure the work they need to accomplish. Each completed small task should count towards the final grade, which of course provides its own form of motivation. And for you as the faculty member, there is an additional motive at work: the completed project or research paper you finally receive from each student will almost certainly be of higher quality than had you collected it for the first time as a finished product. You still won't have a stack full of graduate seminar papers to grade; but whatever you do have will be more polished and substantive than had it all been completed during a marathon session the weekend or night before the big due date.
3. Be human.
- Learn your students' names.
- Let them see that you are interested in who they are and what they are doing outside of class.
- Pay attention to body language and (when it seems warranted) ask about it.
- Show your students that you, too, have a life—a family, a pet, a hometown, a dream, a hobby, a favorite food or book, a past, and even a future.
- Require them to come to your office hours, in pairs or a group of three if that helps, so they will have a way to picture you outside class time and will know firsthand that you do in fact exist beyond that narrow slot in their weekly schedules.
4. Remind yourself that this, too, shall pass.
Much of what I've talked about here focuses on the kinds of motivation we can apply from the outside in, and so it plays to the adolescent world which we hope these teens are going to leave behind. But while they're in transition (and at age 18, most are definitely still in transition), I think we can make these external motivators work in our favor. We hope that our freshmen will eventually act like the adults we all want them to become. External motivations can help speed up this process by teaching students how to attend, engage, participate, and reflect the way adults are supposed to do.
5. Motivate yourself, too.
Work with other colleagues in a variety of disciplines. Talk to them about the students you share. Attend workshops and seminars on your home campus sponsored by your faculty development office. Pay attention to pedagogical research in your discipline. Engage distant colleagues similarly: read newsletters like this one, which often distil the best of what is going on in our classrooms. Or take a look at some of the titles listed at the end of this article for "Further Reading."
6. Last but certainly not least, share your love of learning.
You personally are a wonderful resource for your students. If you love learning and find your discipline thrilling, then say so and show your students why. Like everyone else, students are drawn to genuine expressions of positive regard. Simply "liking" your students would be helpful; but loving your discipline is something you already do. Share the love!
Acknowledgements: I have many wonderful colleagues on the faculty at Northridge.
My thanks to all of them for teaching me about learning.
For Further Reading:
Erickson, B. L., Peters, C. B., & Strommer, D.W. (2006). Teaching first-year college students. San Francisco, CA:Jossey-Bass.
Leamnson, R. (1999). Thinking about teaching and learning: Developing habits of learning with first year college and university students. Sterling, VA:Stylus.
Romack, J. (September 2006). Enhancing students' readiness to learn. The Teaching Professor. Magna, WI: Madison.
November 3, 2008
Discipline Research Project
Round #2, RFPs due
November 7, 2008
ITL's Student Success Train-the-Trainer Workshop.
Please see your campus Faculty
Development Center director.
November 17 (week of), 2008
Discipline Research Project, Round #2, awards announced
Faculty Development Council Meeting, CSU Fullerton
December 12, 2008
Discipline Research Project, Round #2, Project Directors' Planning Meeting
Access to Excellence
As a strategic system-level plan for our 23 California State Universities, Access to Excellence focuses on the intersection of the CSU with the economic, political, and social environment of the State of California, anticipating what the people of the state will need from the CSU in the next decade, and how best to position the institution to meet those needs.
CSU Institute for Teaching and Learning