CS357: Foundations of Artificial Intelligence - Written Assignment: Does It Matter If Machines Understand? (100 Points)

Purpose, Task, and Criteria

Purpose: To decide, and defend, whether it matters if machines genuinely understand — and to see that the answer changes how you design and deploy real AI systems.

Task: Write a 1000-1500 word argumentative essay that analyzes at least two named philosophical positions, defends your own against its strongest counterargument, and ties it to a concrete deployment implication and one guidance statement for CS357 students.

Criteria: Assessed in four equal parts — philosophical engagement, argument quality and evidence, practical implications, and writing quality and originality; see the rubric below for the full breakdown.

Assignment Goals

The goals of this assignment are:
  1. To engage with foundational philosophical arguments about machine understanding and intelligence by accurately characterizing and critically analyzing at least two named positions
  2. To argue for a specific position using evidence from both philosophy and observed AI behavior, addressing the strongest counterargument rather than dismissing it
  3. To connect the philosophical position to a concrete AI deployment scenario with a specific, non-obvious implication for design or practice
  4. To close with one actionable guidance statement for CS357 students that follows from the argued position and would not follow from the opposing position

Background Reading and References

Please refer to the following readings and examples offering templates to help get you started:

The Assignment

Long before large language models existed, philosophers argued over whether any machine could genuinely understand language, experience consciousness, or be held accountable for its actions. Those arguments are no longer merely academic: the systems we build in this course process natural language, generate explanations, and take actions with real consequences. Whether or not those systems “really” understand turns out to matter — for how we design them, how we deploy them, and what obligations we take on when we do.

This assignment asks you to engage seriously with one of those foundational questions and arrive at a position that has consequences for your practice as an AI practitioner.

How to Approach This Assignment

  • Pick the prompt you find genuinely interesting. This sounds obvious, but it matters: the difference between an essay that earns proficient and one that earns beginning is almost always the difference between a student who is actually trying to answer the question and one who is trying to fill 1,000 words. Read all four prompts before choosing.
  • Argue, don’t summarize. Every paragraph should be doing one of these things: (a) presenting your thesis, (b) presenting a philosophical position that supports or challenges it, (c) objecting to that position, (d) explaining why the objection does or does not succeed, or (e) connecting your argument to a deployment scenario. If a paragraph is just describing what Turing or Searle said, cut it or transform it into analysis.
  • Define your terms early and use them consistently. “Understanding,” “consciousness,” “intelligence,” and “awareness” mean different things in different philosophical traditions. Choose one meaning for each term, state it in your introduction, and use it consistently throughout. Swapping definitions mid-essay is the single most common source of logical errors in philosophy papers.
  • Your conclusion should surprise a reader who read only your introduction. A good philosophical essay does not end where it began — it arrives somewhere the introduction pointed toward but did not fully reveal.

Common Pitfall: Using “AI” and “consciousness” interchangeably without defining your terms. Turing, Searle, and Weizenbaum are all talking about related but distinct things: Turing is asking about behavioral indistinguishability; Searle is asking about intentionality (genuine meaning); Weizenbaum is asking about appropriate delegation (what machines should be asked to do, regardless of what they can do). If you mix these up, your essay will argue past itself. Define what you mean by “understanding” in your first paragraph and stick to it.

Recommended time budget:

  • Reading and annotation: 90 minutes
  • Choosing a prompt and outlining: 30 minutes
  • First draft: 90 minutes
  • Revision (argument tightening, evidence deployment): 45 minutes
  • Final proofread: 15 minutes

What proficient work looks like: A proficient essay takes a position that someone could reasonably disagree with, engages seriously with the strongest objection to that position (not a straw man), and closes with an implication specific enough that a student in CS357 would do something differently in their next project because of it.

Thinking Through the Prompts: A Quick-Start Guide to the Philosophers

Before you choose a prompt, read these three-sentence summaries of each philosopher’s core position. These are not substitutes for reading the texts — they are starting points so you are not beginning cold.

Alan Turing (1950): Turing argues that the question “can machines think?” is too philosophically loaded to be useful. He proposes replacing it with a behavioral test: if a machine can carry on a conversation that a human judge cannot distinguish from a human’s conversation, then the machine is, for practical purposes, thinking. His key move is to say that the inability to distinguish behavior is sufficient — we should not require access to the internal experience, because we do not require that of other humans either.

John Searle (1980): Searle argues that behavioral indistinguishability is not sufficient for understanding. His Chinese Room thought experiment imagines a person in a room who manipulates Chinese symbols according to formal rules, producing outputs that look like fluent Chinese conversation — but the person (and therefore the room) does not understand Chinese. Searle’s claim: syntax (the manipulation of symbols by rules) is not sufficient for semantics (genuine meaning). A system that passes the Turing Test might be doing exactly what the room does — and still understand nothing.

Joseph Weizenbaum (1976): Weizenbaum created ELIZA, one of the first conversational AI programs, and was disturbed by how readily users formed emotional attachments to it. His argument is not primarily about whether machines can understand, but about what we should ask them to do: some human activities — therapy, judgment, care — require genuine human understanding and empathy, and delegating them to machines, even capable ones, is a moral failure regardless of the machine’s internal states. The question is not just “can it?” but “should we?”

Suggested Essay Structure

This structure is a suggestion, not a requirement. Use it if you are unsure how to organize your argument; deviate from it if your argument calls for a different shape.

  1. Introduction (~150 words): State your thesis (the position you will argue for) and the scope of your essay (which prompt, which philosophers you will engage with). Do not start with “Since the dawn of time…” or a dictionary definition. Start with the tension that makes your question interesting.
  2. Position A — with objection (~250 words): Present the first philosophical position you are engaging with. Characterize it accurately. Then raise the strongest objection to it. Explain whether the objection succeeds.
  3. Position B — with objection (~250 words): Present the second philosophical position. Characterize it accurately. Then raise the strongest objection to it. Explain whether the objection succeeds.
  4. Your position (~200 words): State which position you find more convincing, and why. Engage with the strongest argument on the other side — do not just dismiss it.
  5. Practical implication (~100 words): Connect your position to a specific AI deployment scenario. What would a developer or deployer do differently if they accepted your position?
  6. Conclusion (~100 words): Restate your thesis in light of the argument you have made. Close with the one actionable implication for CS357 students.

What NOT to Do

  • Do not write a summary of the readings. If your essay could be replaced by a Wikipedia article about Searle or Turing, it is not doing the required philosophical work. Every paragraph must be advancing your argument, not reporting what others argued.
  • Do not conclude “both sides have merit.” This is the most common way to avoid doing philosophy. You must arrive at a position — a claim that someone could reasonably disagree with. “It depends on context” is also not a position unless you specify the conditions under which each conclusion holds and explain why those conditions matter more than the others.
  • Do not use AI to generate your argument. An AI-generated philosophical argument will look like a competent summary of the positions and a hedge at the end. It will not look like a student who has genuinely wrestled with a question and arrived at a view. The rubric specifically rewards originality of argument — not originality of prose style, but originality of the position and the reasoning behind it.

Instructions

Write a 1000–1500 word essay (excluding references) on one of the following prompts. Choose the prompt you find most genuinely interesting; the best essays come from writers who actually want to answer the question.

Prompt A

“Searle’s Chinese Room argument decisively shows that large language models do not understand language. Even if that is true, does it matter for how we deploy them?”

Engage with Searle’s original argument and at least one serious objection to it (for example, the Systems Reply or the Robot Reply). Then shift the question: grant for the sake of argument that Searle is right and LLMs do not understand. What follows for deployment? Does a system need to understand in order to be useful, trustworthy, or dangerous? Arrive at a position.

Prompt B

“The ELIZA effect — the human tendency to anthropomorphize AI systems — is not a bug but a feature: it makes AI systems more usable and more effective. Evaluate this claim.”

Engage with Weizenbaum’s original concern about ELIZA and the broader literature on anthropomorphism in HCI and AI. Consider: what does “more usable” mean and for whom? Are there tasks where anthropomorphism improves outcomes, and tasks where it is actively dangerous? Arrive at a position that does not simply say “it depends” — identify the conditions under which each conclusion holds and explain why those conditions matter more than the others.

Prompt C

“If we cannot determine from behavior alone whether an AI system is conscious or sentient, what ethical stance should we take toward it?”

Engage with the problem of other minds as it applies to AI systems, and with at least one framework for moral status (for example: sentience-based, interests-based, relational). Consider the asymmetry between the cost of wrongly treating a non-conscious system as conscious versus wrongly treating a conscious system as non-conscious. Arrive at a position and defend it against the strongest objection.

Prompt D

“Accountability requires understanding. Therefore, AI agents cannot be held accountable for their actions. Evaluate.”

Engage with what accountability requires — intent, understanding, or something else — drawing on at least one philosophical source and at least one AI deployment scenario where the question of accountability arose (for example: autonomous vehicle liability, content moderation errors, or AI-assisted medical decisions). Does the premise hold? Does the conclusion follow? Who is accountable if the agent is not, and is that answer satisfying?

Requirements

Whichever prompt you choose, your essay must:

  1. Engage with at least two named philosophical positions or arguments, with citation. You may draw from the required readings or from other sources you find; sources must be real and retrievable.
  2. Connect the philosophical argument to at least one concrete AI deployment scenario — a real system, a realistic hypothetical, or a case from the news. The connection must be substantive, not decorative.
  3. Arrive at a position. “Both sides have merit” is not a position. Your conclusion must be a claim that someone could reasonably disagree with.
  4. Close with one implication for CS357 students: one thing a student building or deploying an AI system should do differently because of the position you have argued for.

Common Mistakes

  • Using “AI” and “consciousness” interchangeably without defining your terms. These are different concepts. Define what you mean by “understanding” in your first paragraph and hold to that definition.
  • Summarizing the philosophers rather than arguing with them. The rubric rewards philosophical engagement, which means raising objections, explaining why they succeed or fail, and arriving at a view. Description earns beginning; analysis earns proficient.
  • Concluding “both sides have merit” or “it depends.” These are placeholders, not positions. Force yourself to make a claim someone could disagree with.
  • Writing a practical implication that is trivially obvious. “We should be careful when deploying AI in high-stakes contexts” is not an implication — it follows from almost any philosophical position. Your implication should be specific enough that it would lead to a different decision than the opposing position would recommend.
  • Deploying evidence decoratively rather than precisely. Quoting Searle in your introduction and Turing in your conclusion without connecting them to your argument earns beginning. Quoting a specific passage and then explaining exactly how it supports or undermines a specific claim in your argument earns proficient.

A Note on AI Use

You may use AI tools for brainstorming or grammar checking, but the argument must be your own. An essay that summarizes AI-generated positions without genuine engagement will not satisfy the philosophical engagement or originality criteria. If you use an AI tool, note it briefly at the end of your essay and describe how you used it.

Submission Instructions

Submit a single PDF. Include a references section at the end (not counted in the word count). Word count must appear at the top of the first page.

  • Name the dialectic move you made in this essay: which objection did you allow to genuinely threaten your thesis, and what changed — in your position, your argument, or its scope — as a result of taking that objection seriously?
  • If collaboration with a buddy was permitted, did you work with a buddy on this assignment? If so, who? If not, do you certify that this submission represents your own original work? Please identify any and all portions of your submission that were not originally written by you.
  • Approximately how many hours did it take you to finish this assignment (I will not judge you for this at all…I am simply using it to gauge if the assignments are too easy or hard)?

Submission

In your submission, please include answers to any questions asked on the assignment page, as well as the questions listed below, in your README file. If you wrote code as part of this assignment, please describe your design, approach, and implementation in a separate document prepared using a word processor or typesetting program such as LaTeX. This document should include specific instructions on how to build and run your code, and a description of each code module or function that you created suitable for re-use by a colleague. In your README, please include answers to the following questions:
  • Describe what you did, how you did it, what challenges you encountered, and how you solved them.
  • Please answer any questions found throughout the narrative of this assignment.
  • If collaboration with a buddy was permitted, did you work with a buddy on this assignment? If so, who? If not, do you certify that this submission represents your own original work?
  • Please identify any and all portions of your submission that were not originally written by you (for example, code originally written by your buddy, or anything taken or adapted from a non-classroom resource). It is always OK to use your textbook and instructor notes; however, you are certifying that any portions not designated as coming from an outside person or source are your own original work.
  • Approximately how many hours it took you to finish this assignment (I will not judge you for this at all...I am simply using it to gauge if the assignments are too easy or hard)?
  • Your overall impression of the assignment. Did you love it, hate it, or were you neutral? One word answers are fine, but if you have any suggestions for the future let me know.
  • Using the grading specifications on this page, discuss briefly the grade you would give yourself and why. Discuss each item in the grading specification.
  • Any other concerns that you have. For instance, if you have a bug that you were unable to solve but you made progress, write that here. The more you articulate the problem the more partial credit you will receive (it is fine to leave this blank).

Assignment Rubric

Description Pre-Emerging (< 50%) Beginning (50%) Progressing (85%) Proficient (100%)
Philosophical Engagement (25%) Named philosophical positions are absent or mischaracterized — for example, describing the Chinese Room as an argument about consciousness rather than intentionality At least one named position is cited but engagement is superficial — the essay restates what the philosopher argued without raising or evaluating any objection At least two named philosophical positions are cited and analyzed with reasonable accuracy; the essay raises at least one objection to one position but does not explain whether the objection succeeds and why At least two named philosophical positions are accurately characterized and critically analyzed — for each, the essay raises the strongest objection (for example, the Systems Reply or the Robot Reply for Searle) and explains specifically why the objection does or does not succeed in the student's view, with the student's reasoning, not a summary of secondary sources
Argument Quality and Evidence (25%) The essay does not arrive at a position, or the position is contradicted by the evidence presented A position is stated but the supporting argument is a string of assertions — claims are made without citations or logical connectives between them The essay arrives at a clear position with a coherent argument; at least one passage from a required reading is cited, but it is used decoratively (quoted in the introduction or conclusion) rather than deployed to support a specific claim in the body The essay arrives at a clear, specific position defended by a logically structured argument; at least two specific passages from the required readings are cited at the point in the argument where they are needed (not decoratively); the strongest counterargument to the student's position is identified, stated fairly, and addressed — not dismissed with a single sentence
Practical Implications (25%) No connection is made between the philosophical argument and AI deployment or design A connection to a deployment scenario is mentioned in passing but not developed — for example, "this has implications for how we build AI" without specifying what those implications are The philosophical position is connected to a named, concrete AI deployment scenario (a real system or a realistic hypothetical) with a plausible implication for design or practice, but the implication would follow from almost any philosophical position rather than specifically from the one argued The philosophical position is connected to a named, concrete AI deployment scenario with a specific, non-obvious implication — one that would lead to a different design or deployment decision than the opposing position would recommend; for example, if the essay argues Searle is right that LLMs do not understand, the implication might be a specific requirement for human interpretive oversight that a Turing-compatible view would not require; the essay closes with one actionable guidance statement addressed to CS357 students that a student could act on in their next project
Writing Quality and Originality (25%) The essay is largely a paraphrase of the readings or of AI-generated summary, with no original analytical moves The essay contains original sentences but the argument structure mirrors a single source — the essay essentially reports one philosopher's position and slightly agrees or disagrees at the end The essay is largely original with a discernible thesis, organized body, and conclusion; prose is readable though some passages are imprecise or the thesis is not stated until the second or third paragraph The essay states a specific, arguable thesis in the first paragraph; every subsequent paragraph advances the argument rather than summarizing; the conclusion arrives somewhere the introduction pointed toward but did not fully reveal; prose is precisely written at the right level of technicality for a CS audience without a philosophy background; the word count is between 1000 and 1500 words excluding references, and the count appears at the top of the first page

Please refer to the Style Guide for code quality examples and guidelines.