Claude 3.5 Sonnet

The model and the workspace around it

Anthropic introduced Artifacts alongside Claude 3.5 Sonnet: generated documents, code and designs could be displayed beside the conversation. This is an important distinction for the timeline. The model produces material; the product determines how conveniently a person can inspect and revise it.

Imagine two assistants generating the same page. One returns a long block of code; the other lets you inspect a preview and request changes. The second experience might save time without proving that its underlying model has a higher score on every test. Comparing products means evaluating that experience as well as the model’s output.

Anthropic’s original announcement

A website example with real acceptance criteria

For an illustrative project, request a booking form for a fictional photography business. Specify a name field, email, preferred date and confirmation message. Start by inspecting the page on a narrow screen. Then use only the keyboard, submit empty fields and try a malformed email address. A polished screenshot does not answer any of those questions.

Next, decide what “submitted” means. Does the form only display a message, or does it save a request somewhere? This difference is easy to miss in an impressive demo. Before trusting generated software, define the behavior a person should experience and verify that behavior. These are proposed checks, not results from our own Claude test.

Keep the release version attached to the claim

This entry covers the original June 2024 release. If a comparison uses a later model with a similar name, it needs its own record. Otherwise a historical chart can accidentally make early versions look more capable by giving them features or scores that arrived months later.

For coding results, record the repository, the problem, the tools and the number of attempts. A model that suggests a correct edit in a chat window is being tested differently from an agent that searches files, runs tests and retries. Both can be useful, but the result describes the whole evaluation setup. The benchmark name alone is not enough to establish comparability.