View on GitHub

CS357

Foundations of Artificial Intelligence

CS357: Foundations of Artificial Intelligence - Structured Peer Review

Purpose

To use the two review instruments that run through the whole Project Thread: SQR cards and team check-ins, so review happens before the grade rather than after.

About This Tutorial

Professionals do not wait for the final grade to find out what is wrong with their work; they build review into the process. Today you learn the two review instruments used across the entire Project Thread: SQR cards for reviewing other teams’ artifacts, and the structured check-in for giving private, candid feedback about your own team. We move from the inter-group review cycle → the intra-group check-in → how to receive feedback without flinching.

The 15-Minute SQR Protocol Card (in-class version)

This is the complete run-order for the in-class exchange, self-contained, so you can run it from this card alone:

Minutes Step
2 Silent read of the other team’s artifact. No discussion yet; form your own impressions first.
5 Write your card: one Strength with evidence (point to the sentence or section and say what it accomplishes), one genuine Question (something you actually do not know), and one Risk with a suggested mitigation (no risk without an exit).
5 Exchange and discuss cards with the other team; restate their feedback before responding to it.
3 Log what your team will change as a result: who, what, by when.

The full activity below is required pre-class reading before the first exchange, the stakeholder-brief exchange (see the course schedule); it explains what makes each field of the card work, and how to receive the card you get.

Purpose, Task, and Criteria

Following the TILT (Transparency in Learning and Teaching) framework (tilthighered.com):

   
Purpose To make feedback a routine, structured, low-drama part of teamwork (Goal 13) and to practice communicating critique across team boundaries in a form the receiving audience can actually use (Goal 14), while keeping every team honest with itself through private check-ins.
Task Learn the SQR protocol and the check-in form through three models, then practice SQR on a real brief excerpt in the Exercises.
Criteria You can write an SQR card whose Strength cites evidence, whose Question is genuine, and whose Risk carries a mitigation; and you can complete a check-in that is honest, behavioral, and includes both an appreciation and a request.

Key Concepts

Term Plain-English Definition Where You’ll Meet It
SQR Card A structured peer-review card with exactly three parts: one concrete Strength with evidence, one genuine Question, one Risk with a suggested mitigation. Model 1, and the Exercises
Review cycle The repeating Project Thread pattern: artifact -> peer review -> revise -> present. Feedback arrives while revision is still possible. The stakeholder-brief exchange, proposal critique, and gallery walk, the review milestones on the course schedule
Check-in A short, private, structured form about your own team (contribution, reliability, communication, one appreciation, one request) submitted to the instructor only. Model 2; due at the stakeholder-brief exchange, proposal review, and gallery-walk milestones on the course schedule
CATME-informed feedback Feedback dimensions drawn from the CATME Smarter Teamwork research program (catme.org): behaviorally anchored ratings of how teammates contribute, not personality judgments. The check-in form’s categories
Genuine question A question you actually do not know the answer to, as opposed to criticism wearing a question mark (“did you even consider…?”). Model 1, question 2
Repair move The four-step response to a rupture: name it, own your part, restate the other side, propose a next step. Introduced in the Team Charter activity; used here for receiving hard feedback. Model 3
Psychological safety The shared belief that the team is safe for interpersonal risk-taking (Edmondson, 1999). Structured review protects it by aiming feedback at artifacts and behaviors, never at persons. Everywhere

Part I: Reviewing Other Teams

The Inter-Group Cycle and the SQR Card

Why this matters: Unstructured peer feedback fails in two opposite ways: it is either too kind to be useful (“looks great!”) or too vague to act on (“kind of confusing?”). The SQR card prevents both by forcing exactly three things, each with a job:

Part What It Must Contain What It Must NOT Be
Strength One concrete thing that works, with evidence: point to the sentence, section, or design choice and say what it accomplishes. Generic praise (“well written!”) with no location
Question One genuine question: something you actually wondered while reading, whose answer would improve the work. Criticism disguised as a question
Risk One way this could fail or mislead, with a suggested mitigation: you may not name a risk without offering a way out. A complaint with no exit

The card travels through the Project Thread’s standing review cycle (artifact -> peer review -> revise -> present) at three scheduled moments:

  1. Stakeholder Brief exchange (see the course schedule): teams swap briefs; each team writes SQR cards on the brief it receives; authors revise before the brief feeds the Literature Review.
  2. Proposal cross-team critique (see the course schedule): final-project proposals are exchanged the same way, while there is still time to change course.
  3. Gallery walk (see the course schedule): every visitor fills out an SQR card for every team visited; the receiving team triages all cards into fix before submission / disclose in the report / defer to future work.

The mnemonic: host honestly, walk generously. When you host, show the weak parts too. When you walk, an SQR card is a gift that costs you thought; give real ones.

Questions to Work Through

  1. Here are three “Strength” entries written about the same brief. Rank them by usefulness to the receiving team and justify your ranking: (a) “Great job, very professional!” (b) “The problem statement is strong.” (c) “The problem statement (section 4) quotes the stakeholder’s own words (‘we lose a week every audit season’) which makes the scope checkable against a real cost.”

Hint: Ask of each: could the receiving team learn what to keep doing from this? (a) locates nothing. (b) locates a section but not what works about it. (c) names the location, the technique, and why it works; the team can now repeat the technique elsewhere.

  1. Rewrite this fake question as a genuine one: “Did you even think about whether the registrar’s office would actually use this?” What changed, and why does the fake version damage the review relationship while the genuine version strengthens it?

Hint: A genuine version might be: “What did the registrar say when you asked how this would fit their current workflow? We couldn’t find it in the brief.” The fake version asserts a failure and dares the team to deny it; the genuine version admits what the reviewer doesn’t know and points at a checkable gap. One produces defensiveness, the other produces a to-do item.

  1. Why does the SQR protocol require a mitigation to accompany every Risk? Consider what a review full of unmitigated risks does to the receiving team’s ability to act, and what proposing a mitigation forces the reviewer to do first.

Hint: Ten risks with no exits produce paralysis and a sense of being graded rather than helped. And to propose a mitigation, the reviewer must first understand the team’s constraints well enough to suggest something feasible; the requirement quietly forces the reviewer to actually read the work.

Which of the following is a valid Risk entry on an SQR card?

Answer

“The brief’s problem statement assumes the office will share its spreadsheet data; if that access falls through, the project has no data source; consider asking the stakeholder about a de-identified export at the follow-up, and naming a fallback in the proposal.”

Part II: Reviewing Your Own Team

The Intra-Group Check-In

Why this matters: Inter-group review keeps the work healthy; intra-group review keeps the team healthy. The check-in is a short structured form, informed by the CATME Smarter Teamwork research on behaviorally anchored peer evaluation (catme.org), that every member completes privately, to the instructor only, at three points in the semester (just after the stakeholder-brief exchange, the proposal review, and the gallery walk on the course schedule) deliberately timed after the thread’s crunch points, when memories of how the team actually behaved are fresh.

The form, per teammate (and for yourself):

Dimension The Question Anchored Example Answers
Contribution What has this person actually produced or driven since the last check-in? “Drafted brief sections 1 and 3; ran both interviews” / “Attended meetings but has produced no artifact I can point to”
Reliability When this person commits to something, what happens? “Delivers early and flags risks ahead of time” / “Delivers, but only after reminders” / “Commitments quietly vanish”
Communication How does information flow to and from this person? “Responds within our charter’s 24h norm; surfaces blockers in standup” / “Goes silent under pressure; we learn about problems at the deadline”
One appreciation One specific thing this person did that you want them to keep doing. “When the synthesis stalled, you proposed the outline that unstuck us.”
One request One specific behavior change that would help the team. “Please post ‘seen, will respond tomorrow’ instead of silence; the silence reads as absence.”

Rules that make it work: answers describe behaviors and artifacts, never character (“delivered X late twice,” not “is lazy”); it is private to the instructor; it is how the instructor sees inside teams without ambushes, and it feeds the individual contribution component of the final grade; and it is not a substitute for talking to your team; a request that appears in a check-in but has never been said aloud (kindly, via a repair move) is feedback the teammate never got a chance to act on.

Alongside the private check-in, one public instrument continues: every progress report carries all members’ signatures, re-affirming the charter each time. A member who will not sign is a signal the instructor needs before the deadline, not after.

Questions to Work Through

  1. The check-ins are scheduled just after the stakeholder-brief exchange, the proposal review, and the gallery walk. What would be lost if there were only one check-in, at the gallery walk? Name two distinct failure patterns the earlier check-ins can catch while they are still fixable.

Hint: Pattern one: workload asymmetry that starts small early in the project and compounds, visible at the first check-in, entrenched by the last. Pattern two: the silent member drifting away (or the dominant member crowding others out): at the first check-in this is a conversation; by the gallery walk it is a grade dispute. Feedback has a half-life; the final check-in can only document, not repair.

  1. Why must the “one request” be a behavior (“post your section by Thursday standup”) rather than a trait (“be more responsible”)? Connect your answer to both Edmondson’s (1999) psychological safety and to plain practicality: which of the two can a teammate actually comply with by next week?

Hint: A trait judgment is an attack on identity; it triggers defense, not change, and it teaches the team that check-ins are where you get character-assassinated (there goes psychological safety). A behavior request has a built-in success condition: either the section shows up Thursday or it doesn’t. You can comply with a behavior; you can only argue with a verdict.

  1. You are filling in the check-in and realize your answer about a teammate’s reliability is negative, and you have never raised the issue with them directly. What does the charter (and Model 3’s repair moves) say you should do in the week after submitting the check-in, and why does the private channel not discharge that obligation?

Hint: The check-in informs the instructor; it does not inform the teammate. If the first time they hear about the problem is in an instructor conversation, they were denied the cheap, early, face-saving chance to fix it, which is what the charter’s conflict protocol exists to provide. The check-in AND the direct repair move are both required; each does a job the other cannot.

The structured check-in at the three scheduled milestones is:

Answer

Private to the instructor, and used as one input to the individual-contribution component of grading

Part III: Receiving Feedback

Receiving Feedback Well

Why this matters: Review protocols fail at the moment of reception: a defensive reaction teaches reviewers to soften future feedback, and softened feedback is useless. Receiving well is a learnable skill with a small move set: the same repair moves your charter names (name it, own your part, restate the other side, propose a next step), adapted for feedback:

Move At the moment of feedback, it sounds like…
Name it “That one stings a little; give me a second. It’s also fair.” (Naming your reaction defuses it; pretending you have no reaction leaks anyway.)
Own your part “You’re right that section 3 doesn’t say where the data comes from; I wrote it and I assumed access we haven’t confirmed.”
Restate the other side “So your risk is: if the office won’t share the spreadsheet, we have no evaluation data at all. Did I get that right?” (Restating before rebutting is the single highest-value habit in this table.)
Propose a next step “We’ll ask about a de-identified export in Friday’s follow-up and add a fallback to the proposal. Recorder, can you log that?”

And three norms that keep the room safe for candid review, whichever side of it you are on: thank the reviewer before triaging (even for the card you will ultimately reject; the thanks pays for the next candid card); triage in the open (fix / disclose / defer, logged, so reviewers see their cards mattered); and critique the artifact, receive as the team: a card about “the brief’s section 3” is never about section 3’s author, and the team answers as one.

Questions to Work Through

  1. A team receives an SQR card whose Risk is real but whose suggested mitigation is infeasible (it assumes budget the team does not have). Walk the team’s best response through the four moves. Which move is doing the most work in preventing this from becoming “reviewers just don’t get our project”?

Hint: Restating is doing the heavy lifting: “the risk you see is X” separates the (valid) risk from the (infeasible) mitigation, so the team can accept one without the other. Teams that skip restating tend to reject the whole card because its weakest part was weak, and lose the diagnosis along with the prescription.

Exercises

Practice SQR on a sample brief excerpt (15 minutes). Individually write one complete SQR card on the excerpt below; then compare cards within your team, and the Recorder posts the team’s best S, best Q, and best R (they may come from different cards) to the discussion board.

Excerpt from a sample Stakeholder Brief (section 2 and 4, abridged):

The issue in the stakeholder’s own terms. The coordinator of the campus food pantry told us the pantry “runs on one spreadsheet and whoever remembers.” Donations arrive unpredictably; expiration dates are checked by hand “when someone has an hour, which is never.” Their words: “Twice last year we threw out a whole shelf because nobody caught the dates. That’s someone’s groceries.”

Problem statement. The pantry needs a way to track incoming donations and get ahead of expirations. We propose an agent system that will completely automate the pantry’s inventory, ordering, and volunteer scheduling, eliminating manual work. The coordinator’s spreadsheet will be replaced in the first sprint. This will solve the pantry’s problems and can be reused by every pantry in the county.

You’ve succeeded when: your Strength points at a specific sentence and says what it accomplishes (there is a strong move in the excerpt’s use of quotes); your Question is something you actually cannot answer from the text; and your Risk names a specific failure with a feasible mitigation. If you are stuck on R, look at the distance between what the stakeholder asked for and what the problem statement promises, and at the phrase “replaced in the first sprint.”

Optional stretch: draft the check-in you would write about yourself this week: contribution, reliability, communication, one appreciation, one request. Nobody sees it; the point is to notice how it feels from the other side before the first check-in.

Reflection Prompt

In your notebook, keyed to the Open Questions (Goal 15). How should we live together? Candid review is a social contract: everyone’s work improves only if everyone risks candor and everyone receives it generously. Where else in your life does that contract exist (a rehearsal room, a code review, a kitchen, a team)? Where has it broken, and which side broke first, the candor or the generosity? What will I do? Which is harder for you personally: writing the candid Risk, or hearing it? Name one concrete thing from Model 3 you will try at the stakeholder-brief exchange.

Where This Goes Next

You will use SQR cards for real at the Stakeholder Brief exchange on the course schedule, and your first private check-in is due at the same milestone. The cycle (artifact, review, revise, present) then repeats at the proposal critique and the gallery walk, all the way to Demo Day.

Further Reading