CS377: Database Design - Redis: In-Memory Data Structures and Caching

Activity Goals

The goals of this activity are:

1. To describe the Redis key-value data model and its core data structures (strings, hashes, lists, sets, sorted sets)
2. To apply caching patterns (cache-aside, expiration with TTL) using Redis alongside a relational database
3. To decide when an in-memory key-value store is a better fit than a relational database, and when it is not

Supplemental Reading

Feel free to visit these resources for supplemental background reading material.

• Redis Data Types Documentation
• redis-py Client Library
• Setting up Redis

The Activity

Directions

Consider the activity models and answer the questions provided. First reflect on these questions on your own briefly, before discussing and comparing your thoughts with your group. Appoint one member of your group to discuss your findings with the class, and the rest of the group should help that member prepare their response. Answer each question individually from the activity, and compare with your group to prepare for our whole-class discussion. After class, think about the questions in the reflective prompt and respond to those individually in your notebook. Report out on areas of disagreement or items for which you and your group identified alternative approaches. Write down and report out questions you encountered along the way for group discussion.

---

Model 1: The Redis Data Model

Redis core data structures: every value in Redis is one of these, looked up by a single string key

Structure	Looks like (Python)	Example key	Example value	Typical use
String	str / int	page:home:hits	"5821"	counters, cached HTML/JSON
Hash	dict	user:1001	{name: "Ada", role: "admin"}	one record's fields
List	list (deque)	queue:emails	[job3, job2, job1]	queues, recent-items feeds
Set	set	post:42:likers	{1001, 1002, 1017}	uniqueness, membership tests
Sorted set	dict of member→score, kept ordered	leaderboard	{ada: 980, alan: 870}	rankings, priority queues

Questions

1. Redis has no tables, no columns, and no joins: just keys mapped to values. Colons in key names (like user:1001) are only a naming convention. How would you store the employees and departments data from our join activity in Redis? What operation replaces the join?
2. Which Redis structure would you choose for each of the following, and why: a view counter for each page of a site; the set of students enrolled in a course; a game's top-10 high score list; a work queue of thumbnails to generate?
3. Each structure supports operations that are atomic on the server (e.g., INCR, SADD, ZINCRBY). Why does it matter that INCR is a single atomic operation rather than a GET followed by a SET from your program?

---

Model 2: The Cache-Aside Pattern and TTL

def get_user(user_id):
    key = "user:" + str(user_id)
    cached = r.hgetall(key)          # 1. try the cache
    if cached:
        return cached                # 2. hit: microseconds, no DB touched
    row = query_relational_db(user_id)   # 3. miss: ask the database
    r.hset(key, mapping=row)         # 4. fill the cache for next time
    r.expire(key, 300)               # 5. ...but only for 300 seconds (TTL)
    return row

Questions

1. Trace two consecutive calls to get_user(1001): which steps run on the first call, and which on the second? Where did the second call's answer come from?
2. Step 5 sets a time-to-live (TTL): Redis deletes the key automatically after 300 seconds. What could go wrong if we cached forever (no TTL) and someone updated the user's name in the relational database? This is the classic stale cache problem.
3. Suppose the user is updated through our own application. Besides waiting for the TTL to expire, what could the update code do to the cache to keep readers consistent? (Hint: DEL user:1001 — this is called cache invalidation.)
4. Redis keeps data in memory, so a crash can lose recent writes (unless persistence is configured). Why is that an acceptable risk for a cache, but not for the system of record?

Embedded Code Environment

You can try out these code examples in a development environment of your choice! Note that some embedded projects have multiple source files; you can see those by clicking the appropriate file tab to open that file.

---

Why Redis?

A relational database keeps your data safe, consistent, and richly queryable — but every query pays for that power: parsing SQL, planning, reading pages from disk, checking transactions. Many real workloads ask the same simple question thousands of times per second (“what’s user 1001’s profile?”, “how many likes does this post have?”). Redis (REmote DIctionary Server) attacks exactly that case: it is a key-value store that keeps everything in memory, answering simple lookups in well under a millisecond. Think of it as a giant, shared, network-accessible Python dictionary — with superpowers.

Redis is one of the NoSQL data models from our NoSQL Data Models activity (the key-value model, with graph extensions used in the Redis lab). This activity focuses on its two most common day-job roles: a data-structure server and a cache in front of a relational database.

Redis as a Data-Structure Server

Unlike a plain key-value store, a Redis value is not just a blob — it can be a string, hash, list, set, or sorted set, each with atomic server-side operations. A quick tour in Python using redis-py (pip install redis):

import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)

# Strings: values and counters
r.set("page:home:hits", 0)
r.incr("page:home:hits")            # atomic increment -> 1

# Hashes: a record's fields under one key (like a small dict)
r.hset("user:1001", mapping={"name": "Ada", "role": "admin"})
r.hget("user:1001", "name")         # -> "Ada"

# Lists: push/pop from either end (queues and feeds)
r.lpush("queue:emails", "job1", "job2")
r.rpop("queue:emails")              # -> "job1"  (FIFO with LPUSH+RPOP)

# Sets: uniqueness for free
r.sadd("post:42:likers", 1001, 1002, 1001)
r.scard("post:42:likers")           # -> 2 (the duplicate was ignored)

# Sorted sets: members ordered by a numeric score
r.zincrby("leaderboard", 100, "ada")
r.zrevrange("leaderboard", 0, 9, withscores=True)   # top ten

The expected output of the annotated lines, run in order:

Call	Result
r.incr("page:home:hits")	1
r.hget("user:1001", "name")	"Ada"
r.rpop("queue:emails")	"job1"
r.scard("post:42:likers")	2
r.zrevrange("leaderboard", 0, 9, withscores=True)	[("ada", 100.0)]

Notice what’s missing: no schema declaration, no types, no relationships. Structure lives in your key-naming conventions (user:1001, post:42:likers) and in your program’s discipline — the flexibility and the danger of the key-value model in one stroke.

Redis as a Cache: The Cache-Aside Pattern

The most common production use of Redis is as a cache in front of a slower system of record:

             hit (fast path, ~0.1 ms)
  client ---> Redis ------------------------------.
                |  miss                            \
                v                                   v
          relational DB --(row)--> store in Redis --> client
             (system of record, ~10-100 ms)

The recipe, called cache-aside, is always the same five numbered steps shown in the model above: try the cache; on a hit, return it; on a miss, query the database; fill the cache; set a TTL.

Expiration (TTL) in Practice

r.set("weather:collegeville", '{"tempF": 71}', ex=600)  # expire in 10 minutes
r.ttl("weather:collegeville")   # -> 600 (seconds remaining)
# ...ten minutes later...
r.get("weather:collegeville")   # -> None: Redis deleted it for us

The TTL is your freshness dial: shorter TTLs mean fresher data but more database traffic; longer TTLs mean faster responses but staler reads. When your own application performs the update, it can do better than waiting: invalidate (r.delete("user:1001")) or overwrite the cached entry as part of the write path. As the saying goes, cache invalidation is one of the two hard problems in computer science — always ask “who deletes this key when the truth changes?”

When Redis, and When a Relational Database?

Question to ask	Points to Redis	Points to relational
Is losing recent writes tolerable?	Yes (cache, session data, counters)	No (orders, grades, money)
Access pattern?	Lookup by known key	Ad-hoc queries, joins, aggregation
Consistency needs?	Eventual freshness is fine	Transactions and constraints required
Data size vs. memory?	Hot subset fits in RAM	Full dataset, disk-sized
Latency budget?	Sub-millisecond	Milliseconds acceptable

The punchline: it is rarely either/or. The standard architecture keeps the relational database as the durable system of record and uses Redis for the hot, simple, high-frequency reads — the best of both worlds, at the price of managing staleness.

Common Pitfalls

• Treating the cache as the source of truth. If a key might have been evicted, expired, or lost in a restart, your code must be able to rebuild it from the database.
• Caching without a TTL. A key with no expiration and no invalidation path serves stale data forever.
• Unbounded keys. A list or set that only ever grows (e.g., LPUSH with no trim) eventually exhausts memory; pair feeds with LTRIM and let TTLs reap idle keys.
• Storing secrets in code. A hosted Redis instance has a hostname, port, and password: keep them in a .env file, not in your source (see Keeping Credentials Out of Your Code with dotenv).

Submission

I encourage you to submit your answers to the questions (and ask your own questions!) using the Class Activity Questions discussion board. You may also respond to questions or comments made by others, or ask follow-up questions there. Answer any reflective prompt questions in the Reflective Journal section of your OneNote Classroom personal section. You can find the link to the class notebook on the syllabus.