Formative feedback in the Architecture crit
What’s the story?
The ‘crit’ is a common formative assessment technique in many studio-based subjects like Architecture. With close and frequent student-teacher contact in the studio environment, frequent individual tutorials, and regular formal crits offering feed forward on the work in progress, there is ample opportunity for engagement with and dialogue about assessment, as advocated by the Compact. This case study illustrates how even in well-designed assessment and feedback structures, formal evaluation can point to opportunities for improvement. In this case an evaluation of the crit process suggested several possibilities for enhancement, including making more use of peer feedback.
Aspects of the assessment compact scrutinised in this case study
This case study concerns an intervention concentrating on three Compact clauses:
Clause 2.1 – “Assessment is central to the curriculum, and there should be no distinct boundary between assessment, teaching and learning. All academic staff will therefore be encouraged to regard assessment as a fundamental and integral part of programme design, and one that is intended to shape and develop learning, as much as to judge and measure it.”
Clause 2.2 – “Assessment is designed at programme level, to integrate module assessment and ensure that assessment shapes learning in a coherent and holistic fashion, consistent with the aims and learning outcomes of the programme …”
Clause 2.5 – “students are given supportive, constructive and timely feedback as an essential part of their learning … and have a clear sense of what they need to do to improve, with subsequent opportunities provided to act on the feedback and to put the advice given into practice”
What aim(s) did you have in changing your assessment approach?
The ‘crit’ assessment technique is a common to many studio-based subjects and has long been used in Oxford Brookes Architecture. This evaluation aimed to identify both features of the process that work well and opportunities to improve it.
What change(s) did you actually make?
In many ways the studio environment in Architecture and similar subjects represents a ‘gold standard’ of student-tutor feedback opportunities. There are regular opportunities for formative feedback and feed forward built into the design of these modules. In the main these take the form of frequent teacher-student contact in the studio, regular individual tutorials of about 10-15 minutes each, and regular ‘crits’, where for about 15-20 minutes students present progress on their project and obtain feedback and advice for taking their project forward. Usually two staff members participate in the crits and, in theory at least, peers who are present can also contribute their feedback.
However, frequently peer feedback in crits does not happen. Moreover, the duration, complexity and number of simultaneous student design projects and the number of staff involved presents logistical challenges in ensuring that all students can make the most of the opportunities they have in this environment.
The process was evaluated to investigate how it could be improved. The recommendations are listed below.
How did you evaluate this intervention?
An OCSLD researcher analysed data from a module evaluation survey at the end of semester 2, 2011. The primary data analysed was a final, free response item inserted into the usual survey, asking: ‘Have you any comments on the usefulness to you of the crit sessions on your project?’ Many students made responses on other items that were relevant to feedback in general and the crits in particular. These responses were included in the analysis.
There were 99 respondents.
All of the responses to item 5 were transcribed, along with responses to other items relevant to feedback and/or the crit sessions. These transcripts were imported into NVivo for analysis. Each student response was coded:
- as either positive, negative, or neutral/no response to item 5
- by a significant idea that the author wanted to convey about feedback and/or the crits
Thus, a given student comment might be coded in several different categories, depending upon the quantity and complexity of their feedback.
What do students say about this intervention?
Students on this module were very satisfied with the feedback they obtain through their studio work and the crits in particular. On item 5 there were:
- 53 (53 %) unambiguously positive responses
- 17 (17 %) unambiguously negative responses
- 29 (29 %) neutral or gave no response
A majority were complimentary about the crit sessions, with a further substantial number neutral or, at least, without negative views. Typical positive responses are:
- The feedback from the crit is great for helping to develop and assess work
- They are very useful, especially with external critics
- Really useful feedback from crits this year. Enjoy having guest critics.
The negative responses tended to be ‘less negative’ than the positive ones were positive, indicating dissatisfaction with just one or two aspects of the crits, rather than the overall approach. For example,
- A little too informal. Would prefer something with more severity
- The comments at the crit sessions usually tell me what I already know, and rarely give advice which is crucial
- I never got a mark after a crit so did not know where I stood
The significant ideas that emerged were as follows:
21 (21%) of the responses commented explicitly on how the crit sessions helped them to develop their work. These were a subset of the positive responses, about 40% of all the positive responses.
Variety of feedback.
16 respondents commented on gaining feedback from a variety of people; in the main these were the two tutors, plus guests. All but one of these were appreciative of this aspect.
Although, as noted above, students appreciated the way feedback in crits helped them to improve their work, a small but significant proportion of respondents pointed to difficulties about interpreting their feedback. Problems included:
- Conflicting feedback. 12 respondents commented that individual tutors sometimes contradicted each other, or that the feedback from a tutor was internally contradictory.
- Unclear feedback. 12 respondents noted that they had received feedback that in some way or other they could not interpret or act on. For example:
- Unsure about feedback and what tutor want. Prefer a more straightforward feedback.
- Useful but often advice is still quite vague. Need specific advice.
- Sometimes the feedback is quite vague, although intended to let students think and invent things, but not very helpful when you get stuck or lost.
It should be noted here that students’ desire and expectation for precise and straightforward feedback may sometimes be inappropriate if it is to support development of student learning and work in the long run. Students need to interpret their feedback in relation to their own stage of learning and understandings of their subject development. The Compact encourages opportunities for dialogue to be offered. It encourages students to take the initiative to ask questions if they need more advice until they have sufficient understanding of assessment and their subject to be confident in their interpretations.
Also it is worth noting that at least four of the 12 who were concerned about the clarity of their feedback appear to be from students whose first language is not English. It may be worth taking simple measures with such students, like speaking slowly, asking frequently if they know what to do about the remarks they have heard, asking frequently if they need further clarification. Written comments are likely to be particularly important for these students.
- Disagreeing with feedback. Six students expressed disagreement with the feedback or advice they had been given. In such cases, if students treat feedback as a product rather than a dialogic process, they have to choose between going ahead despite advice to the contrary and doing something they don’t agree with or understand. The opportunity for dialogue should exist for students to test their ideas and the Compact encourages students to take responsibility for development of their learning and work.
- Need for improved written feedback. Ten respondents commented on written feedback, ranging from a request for written comments to be ‘more legible,’ to requests for written feedback sheets to be used more often, to requests for better written feedback for record keeping purposes.
Need for encouragement
Five students noted that they would have liked the crits to attend more to encouraging them, or felt that feedback was too critical of their work, e.g.
- I sometimes felt like I could have benefited from some encouragement, rather than just criticism
- Also I found them too critical which did not aid my design work – needs more balance even if you are not the best.
Within this group we find students who find the crit process highly stressful. One said
- I’m sure the pressure would cause hair loss
5 students said they would have appreciated more engagement by their peers in giving feedback in crits, e.g.
- encourage more student engagement
- lack of participation of fellow students in crits
Although the number of comments on this is low, author of this report has observed several crit sessions and has the same feeling as these students: there are opportunities to elicit more peer feedback and interaction, addressing things like self- and peer-assessment that the Compact encourages. Presently peers are a largely untapped resource.
What has been the impact of this change in assessment?
The evaluation of the crit method was undertaken on the basis that the fundamental design is sound and is rich with the assessment and feedback processes advocated by the Compact. From that basis it sought to identify possible improvements to managing the process. The valuation found that the crit sessions are highly valued and students generally feel that they contribute significantly to developing and improving their design projects. There are indications in the data that the mechanics of how they are conducted could be improved in several ways. Recommendations were that studio tutors consider:
- how they might compare notes before they offer feedback to students, to reduce the potential for offering conflicting advice
- methods for engaging peers in crits, including structured peer feedback
- how to improve written elements of feedback
- how to keep records of tutorials so that student progress is tracked and their development is acknowledged and praised in crits
- how to reduce stress and provide more encouragement, especially of struggling students
- methods for better supporting students whose first language is not English in crits and tutorials