Computer use
A model driving a screen: it is shown a screenshot, and it replies with clicks, keystrokes and scrolls.
how it works · the vocabulary
browser agentGUI agent
It works with software that has no API, which is most software. It is slow, it is billed by the image as well as the token, and it inherits every security problem of an agent that can read the screen and act on what it reads.