20.8 What the model knows about the screen
- Approved by
- Panaversity ·
In everyday life. You tell a coworker on the phone, "look at the third line". They see a different page of the report. You misunderstand each other until one of you says which page.
What you will learn. What the model sees when the person uses the screen, and where the screen's state lives.
Why it matters. A person who clicks and then asks a question expects the AI to know what they are looking at. It does not, unless something tells it.
One request, start to finish. At 11:40, Luis moves to the third page. Then he asks, "Why is the second row late?"
- The model last saw page one, in the text answer of the first
list_open_invoicescall. The page's own calls went to the screen, not into the conversation. - It answers about page one's second row, the wrong invoice.
- The builder changes the page. When Luis moves to another page, the page tells the AI app which rows are showing.
- Luis asks again. The model reads the update with his question and answers about the right invoice.

Figure 20.9. The model knows only what it was told. An update from the page tells it what is showing.
How does the screen talk to the model?
Two messages, both from the page to the AI app:1
- A screen update (
ui/update-model-context) gives the model context for its next turn. Each update replaces the last, and the AI app may hold it until the person's next message. Use it for "what is on the screen now". - A chat message (
ui/message) adds a message to the conversation, as if the person wrote it, and the AI app may ask the person first. Use it rarely, for example for an "Ask about this invoice" button.
Where does the screen keep its state?
Nowhere that lasts. The page is drawn again when the chat is reopened, and what it showed came from the connector. That is the third of Chapter 1's four facts: what you keep must be saved to a known place. A filter on a screen is not a record. Anything that must last goes through a tool to the connector's database.
At work. In Lab 20, Task 7, Luis's question about the second row is asked twice, before and after the page sends its update, and the answers are compared.
The operating rule: tell the model what the screen shows when it changes, and keep nothing on the screen that must last.
Check yourself
Question 1 / 3
0 answered
Luis moves to the third page, then asks "Why is the second row late?" The page sends nothing to the AI app. What does the model answer about?
Sources
- Owner
- Panaversity