CS357: Foundations of Artificial Intelligence - Lab: Tools and MCP (100 Points)

Assignment Goals

The goals of this assignment are:

To give an agent real tools using native function calling, and to know which side of the boundary your code owns
To make an agent reason explicitly, and to measure whether the reasoning paid for itself
To both author and consume an MCP server, and to articulate what the protocol standardizes
To constrain model output so that downstream code can parse it reliably rather than hopefully

The Assignment

Lab: Tools and MCP

The Local Agent Lab built an agent that perceives, plans, and acts in a loop - but its only action was producing text. This lab gives it hands.

The three capabilities below were previously bolted onto the Local Agent Lab, where they were due before the sessions that teach them. They now stand on their own, handed out the day we cover tool use and due after we cover MCP, so that every part of this lab is something you have already seen in class.

Prerequisites, all taught before this lab is due: the Tool Use and Function Calling session, the MCP and APIs session, and the structured-output reading attached to both.

The Three Capabilities

Every submission must show that you can make an agent use a tool, make an agent reason, and work with MCP (the Model Context Protocol). Each capability comes in two flavors — build it from scratch (you own the wiring) or drive it from a framework / served model / existing server (you own the configuration). Complete at least one option from Tool Use, at least one from Reasoning, and at least one from MCP. You may do both flavors of one if it interests you, but one of each capability is the floor. Fold your chosen options into your writeup with the transcript evidence each one asks for.

Required for everyone — Structured output. Before your tool-use option can be trusted, the model has to return data your code can parse reliably, not free-form prose that happens to contain JSON. As part of your Tool Use exploration, demonstrate one structured-output technique and show it recovering from a case where naive parsing fails. Pick one:

Ollama’s format parameter — pass a JSON Schema (or "json") in the request so the server constrains the response to valid JSON. See the Ollama structured outputs docs.

Instructor — define a Pydantic model (a typed schema much like the dataclasses you already write) and let Instructor validate and auto-retry until the response conforms.

Outlines — constrain decoding to a grammar/regex/JSON schema so invalid tokens are impossible, not merely discouraged.

Deliver: a two-or-three-sentence note in your writeup showing a before (free-form parse breaks on a real model response) and after (constrained output parses every time), and one sentence on which of the three guarantees validity versus merely encourages it. This is the reliability glue the rest of your agent’s tool-calling depends on.

Tool Use — pick at least one:

Tool Use · From Scratch — expose a function to the model

Give your agent a real, typed tool using native function calling (not the week-1 regex parse). Define a Python function, describe it as a JSON schema in a tools list, and let the model emit a structured tool_calls request that your code executes and feeds back as a tool-role message. Do this against Ollama’s /api/chat or OpenWebUI’s OpenAI-compatible /api/chat/completions — the schema is identical across both (see the Tool Use and Function Calling activity, Parts II–2b). Deliver: your tool schema, a transcript showing the model requesting the tool and your program executing it, and one sentence on what your code — not the model — is responsible for.

Tool Use · From a Framework — give an agent tools you did not wire

Hand the same tool to an agent through a framework so the framework owns the tool-calling loop. Register a Python function as a tool with smolagents (Hugging Face’s lightweight agent library — the gentlest starting point), LangChain/DeepAgents, or Agno, and let it drive invocation (see the Agent Frameworks activity, including how to point the framework at your local Ollama/OpenWebUI model). If you are new to frameworks, prefer smolagents: it is a much thinner wrapper than LangChain, so less of the loop is hidden and the code you write stays close to the from-scratch version. Deliver: the tool registration, a run transcript, and two things the framework hid from you that you had to do by hand in the from-scratch version.

Reasoning / Thinking — pick at least one:

Reasoning · From Scratch — make the agent think, and measure it

Add explicit reasoning to your agent and test whether it helps. Either (a) insert a scratchpad/chain-of-thought step where the model reasons before it answers, or (b) spend test-time compute: sample several reasoning paths at nonzero temperature and select the best (majority vote or a self-check). Run both the plain and the reasoning version over a fixed set of at least eight tasks at a fixed seed, and report the accuracy delta and the extra tokens/latency it cost. Deliver: both versions, the paired results table, and a sentence on when the extra reasoning earned its cost. Concepts are in the model-types lecture.

Reasoning · From a Model/User Perspective — use a reasoning model

Drive reasoning by choosing the model rather than building the loop. Run a reasoning-capable model (or toggle a “think step by step” / extended-thinking mode where your server supports it) and compare it against a direct-answer model on the same eight-task set. Report accuracy, latency, and token cost for each, and identify a task type where the reasoning model clearly wins and one where it is wasteful. Deliver: the comparison table and a short recommendation on which model you would ship for this workload and why. See the model-types lecture for what makes a model a “reasoning” model.

MCP (Model Context Protocol) — pick at least one:

MCP · Create — stand up your own MCP server

Expose your tool(s) over MCP so any MCP-aware client can discover and call them, not just your own loop. Build a small MCP server (e.g. with the Python MCP SDK / FastMCP) that advertises one or two tools, then connect a client and show the discover → invoke round-trip. Deliver: the server code, a transcript of a client listing the tools and calling one, and one sentence on what MCP standardizes that a hand-rolled tools list does not. (If you take the MCP Server with OAuth 2.0 direction, that fully satisfies this option.) Background: the MCP activity and the free Hugging Face MCP Course (built with Anthropic), whose early units walk through building and connecting an MCP server step by step.

MCP · Use — connect your agent to an existing MCP server

Consume MCP instead of authoring it. Point your agent (or a framework client) at an existing MCP server — for example a filesystem, fetch, or SQLite server — and let it discover the server’s tools and call them to complete a task. Deliver: the connection/config, a transcript showing tool discovery and at least one successful invocation, and one sentence on the trust question this raises (you are now running someone else’s tool definitions). Background: the MCP activity.

What to Submit

One repository or archive containing your code, plus a writeup that includes, for each of the three capabilities, the deliverable that capability asks for. Record the model and parameters you used so a reader can reproduce your runs, and include an AI-use disclosure naming what was AI-assisted and how you verified it.

Estimated Effort

Roughly 6 to 8 hours: about 2 hours for tool use plus structured output, 2 to 3 for the reasoning comparison (most of it waiting on runs), and 2 to 3 for MCP. The reasoning comparison is the one to start early, because eight tasks times two conditions is a lot of wall-clock time if you leave it to the last night.

Submission

In your submission, please include answers to any questions asked on the assignment page, as well as the questions listed below, in your README file. If you wrote code as part of this assignment, please describe your design, approach, and implementation in a separate document prepared using a word processor or typesetting program such as LaTeX. This document should include specific instructions on how to build and run your code, and a description of each code module or function that you created suitable for re-use by a colleague. In your README, please include answers to the following questions:

Describe what you did, how you did it, what challenges you encountered, and how you solved them.
Please answer any questions found throughout the narrative of this assignment.
If collaboration with a buddy was permitted, did you work with a buddy on this assignment? If so, who? If not, do you certify that this submission represents your own original work?
Please identify any and all portions of your submission that were not originally written by you (for example, code originally written by your buddy, or anything taken or adapted from a non-classroom resource). It is always OK to use your textbook and instructor notes; however, you are certifying that any portions not designated as coming from an outside person or source are your own original work.
Approximately how many hours it took you to finish this assignment (I will not judge you for this at all...I am simply using it to gauge if the assignments are too easy or hard)?
Your overall impression of the assignment. Did you love it, hate it, or were you neutral? One word answers are fine, but if you have any suggestions for the future let me know.
Using the grading specifications on this page, discuss briefly the grade you would give yourself and why. Discuss each item in the grading specification.

Any other concerns that you have. For instance, if you have a bug that you were unable to solve but you made progress, write that here. The more you articulate the problem the more partial credit you will receive (it is fine to leave this blank).

Assignment Rubric

Description Pre-Emerging (< 50%) Beginning (50%) Progressing (85%) Proficient (100%)

Tool Use (30%) No working tool call, or the model is asked for a tool but nothing executes. A tool is called, but the schema is untyped or the result is not fed back to the model as a tool-role message. A typed tool is registered and invoked end to end, with a transcript showing the round trip. A typed tool is registered and invoked end to end, the transcript shows request and execution and the fed-back result, and the writeup names precisely what your code is responsible for that the model is not.

Structured Output (20%) Output is parsed ad hoc from prose; no technique demonstrated. A structured-output technique is used but no failure case is shown, so the reliability claim is untested. One technique is demonstrated with a before-and-after: naive parsing breaks on a real response, the constrained version does not. As progressing, and the writeup distinguishes which techniques guarantee validity by construction from those that merely encourage it, with the evidence to back the distinction.

Reasoning, Measured (25%) No reasoning variant, or no comparison. A reasoning variant exists but is compared informally, without a fixed task set or a fixed seed. Plain and reasoning versions run over at least eight fixed tasks at a fixed seed, with an accuracy delta reported. As progressing, plus the token and latency cost of the reasoning, and a defensible sentence on when that cost was earned and when it was not.

MCP (20%) No MCP work. An MCP server or client is configured but no discover-and-invoke round trip is shown. A transcript shows tool discovery followed by a successful invocation. As progressing, and the writeup states what MCP standardizes that a hand-rolled tools list does not - and, for the consume option, names the trust question raised by running someone else's tool definitions.

Writeup and Reproducibility (5%) No writeup, or one that cannot be followed. A writeup exists but a reader could not reproduce the runs from it. Model, parameters, and commands are recorded well enough to reproduce. Fully reproducible, with an AI-use disclosure naming what was AI-assisted and how it was verified.

Please refer to the Style Guide for code quality examples and guidelines.

Description	Pre-Emerging (< 50%)	Beginning (50%)	Progressing (85%)	Proficient (100%)
Tool Use (30%)	No working tool call, or the model is asked for a tool but nothing executes.	A tool is called, but the schema is untyped or the result is not fed back to the model as a tool-role message.	A typed tool is registered and invoked end to end, with a transcript showing the round trip.	A typed tool is registered and invoked end to end, the transcript shows request and execution and the fed-back result, and the writeup names precisely what your code is responsible for that the model is not.
Structured Output (20%)	Output is parsed ad hoc from prose; no technique demonstrated.	A structured-output technique is used but no failure case is shown, so the reliability claim is untested.	One technique is demonstrated with a before-and-after: naive parsing breaks on a real response, the constrained version does not.	As progressing, and the writeup distinguishes which techniques guarantee validity by construction from those that merely encourage it, with the evidence to back the distinction.
Reasoning, Measured (25%)	No reasoning variant, or no comparison.	A reasoning variant exists but is compared informally, without a fixed task set or a fixed seed.	Plain and reasoning versions run over at least eight fixed tasks at a fixed seed, with an accuracy delta reported.	As progressing, plus the token and latency cost of the reasoning, and a defensible sentence on when that cost was earned and when it was not.
MCP (20%)	No MCP work.	An MCP server or client is configured but no discover-and-invoke round trip is shown.	A transcript shows tool discovery followed by a successful invocation.	As progressing, and the writeup states what MCP standardizes that a hand-rolled tools list does not - and, for the consume option, names the trust question raised by running someone else's tool definitions.
Writeup and Reproducibility (5%)	No writeup, or one that cannot be followed.	A writeup exists but a reader could not reproduce the runs from it.	Model, parameters, and commands are recorded well enough to reproduce.	Fully reproducible, with an AI-use disclosure naming what was AI-assisted and how it was verified.