Skip to main content
Andrei Moise

On this page

  1. The loop that stops
  2. Asleep until something happens
  3. Surviving a crash
  4. Who goes first
  5. Room in the context
  6. Cheapest tool first
  7. Keeping the cache warm
  8. What it remembers
  9. Training for the product
  10. Two improvement loops
  11. Sources
All notes
5 min readAgentic Workflows

Always-on agents

Notes on Lee Robinson's Stanford CS146S talk, How always-on agents work, with a live widget for each idea.

Part 01

The loop that stops

▶ 00:26 in the talk

Fig 1.1 · Close the lidLive

Try: scroll to play the whole run, laptop first, then the phone

agent — zsh

~/agent $ python agent.py

>

messages in memory: 0esc to interrupt

Tuesday

9:41

No messages from the agent

Press Enter to give the agent its task.

Part 02

Asleep until something happens

▶ 10:12 in the talk

Fig 2.1 · One dayLive

Try: scroll to move through the day, or untick a wake-up

Wake-ups today · each passes the router first

Scroll to move through the daynow 00:00

Agent process · one turn, then stops
Agent's computer · restored only when a tool needs it
Saved state on the server
00:0006:0012:0018:0024:00
5 min
Agent awake
28 min
Computer awake
22 min
Of a 24/7 machine
1.5%
Messages to you
3 of 5

Agent stops after 2 idle min · the computer's memory and disk are snapshotted before it stops

Part 03

Surviving a crash

▶ 16:20 in the talk

Fig 3.1 · Crash a runLive

Try: press Play, then crash at step 4 and make the steps idempotent

The server crashes during step
Job queue · starts the turn again
  1. Read the request
  2. Search flights
  3. Hold a seat
  4. ✕ Email the itinerary
  5. Read the request
  6. Search flights
  7. Hold a seat×2
  8. Email the itinerary×2
  9. Book and pay
Durable workflow · resumes at the crashed step
  1. Read the request
  2. Search flights
  3. Hold a seat
  4. ✕ Email the itinerary
  5. Email the itinerary×2
  6. Book and pay
Queue · steps redone
3
Queue · done twice
2
Durable · steps redone
0
Durable · done twice
1

×2 = a side effect that happens again on the re-run · the crash lands after the side effect, before it is recorded

Part 04

Who goes first

▶ 17:21 in the talk

Fig 4.1 · Send messagesLive

Try: press Play, or send from Team and Slack, then from You

Send a message as
Running turn

Idle

Waiting, first in line on top

Empty

Messages received
0
Turns run
0

Part 05

Room in the context

▶ 19:58 in the talk

Fig 5.1 · Fill the contextLive

Try: connect more tools, then move things out of the conversation

40
Move out of the conversation
Every tool's details up frontdoesn't fit
Names only, details for the 3 in use25% free
Up front · tokens used
241k
On demand · tokens used
150k

Illustrative token counts · window 200k

Part 06

Cheapest tool first

▶ 21:34 in the talk

Fig 6.1 · Pick a taskLive

Try: press Play, or pick a task and break the connector

  1. 1What it already knows · memory and this conversation
  2. 2App connector · an API, fast and exact
  3. 3Web search · public pages
  4. 4Signed-in browser · reads the page structure
  5. 5Full desktop · clicks pixels on a 1280×800 screen
  6. 6Ask you · logins, 2FA codes, payments
cheap, fast↓slow, costly

Answered at rung 1 after trying 1 option.

Part 07

Keeping the cache warm

▶ 27:45 in the talk

Fig 7.1 · Break the cacheLive

Try: press Play, then put the time at the top and play again

Mistakes
Fixes
Tokens cached
89%
Total cost
274k
Last prompt
148k

Illustrative · cost in fresh-token units · a cached token costs 10% of a fresh one · summary after call 10, while the cache is warm

Part 08

What it remembers

▶ 32:51 in the talk

Fig 8.1 · Weeks of memoryLive

Try: scroll to let the weeks pass

week 0
Profile · always in the prompt · 3 here, up to 100 facts in 4,000 characters
  • Works on the payments team9.0
  • Prefers morning meetings8.0
  • Works from the London office7.0
Log · dated events in the prompt · 3 here, the 30 most relevant
  • Shipped the March release6.0
Notes · short-lived details
  • Waiting on a reply from legal4.0
Everything else · found with a memory search tool
  • Writes in British English6.0

Score = importance, halved every 3 weeks for events and every week for notes · memories are information, never instructions

Part 09

Training for the product

▶ 34:32 in the talk

Fig 9.1 · Diagram
  1. Pre-training: learns language, code and general knowledge from a huge amount of text
  2. Supervised fine-tuning: learns this product's tools and formats from worked examples
  3. Reinforcement learning: practises real agent tasks and is scored on the result
Pre-training: learns language, code and general knowledge from a huge amount of text
↓
Supervised fine-tuning: learns this product's tools and formats from worked examples
↓
Reinforcement learning: practises real agent tasks and is scored on the result

Part 10

Two improvement loops

▶ 36:04 in the talk

Fig 10.1 · Turn the loopsLive

Try: press Play, or step each loop

Every day
  1. 1An agent gets a task wrong
  2. 2Turn it into an eval
  3. 3Fix the harness, prompt or inference
  4. 4Check the evals
  5. 5Ship the fix
Every few months
  1. ATrain a new model on the evals
  2. BPut it in the product
  3. CUsers find new uses
  4. DNew behaviour to test
Evals written
0
Fixes shipped
0
Waiting to train on
0
Model
v1

Sources

  1. Lee Robinson, Stanford CS146S: How always-on agents work(opens in a new tab)

Reach out

ai@andreimoise.com