Anthropic released a fascinating new capability today called "Computer Use" - a mode of their Claude 3.5 Sonnet model where it can do things like accept screenshots of a remotely operated computer and send back commands to click on specific coordinates, enter text etc
My notes on what I've figured out so far: simonwillison.net/2024/Oct/22/…
Initial explorations of Anthropic’s new Computer Use capability
Two big announcements from Anthropic today: a new Claude 3.5 Sonnet model and a new API mode that they are calling computer use. (They also pre-announced 3.5 Haiku, but that’s …simonwillison.net
))
miki
in reply to Simon Willison • • •