Skip to main content

Anthropic Vertex AI

info

You need to enable the Vertex AI API and set up the Google Application Default Credentials.

Chat model

We recommend configuring Claude 3.5 Sonnet as your chat model.

config.json
{
"models": [
{
"title": "Claude 3.5 Sonnet",
"provider": "anthropic-vertexai",
"model": "claude-3-5-sonnet-20240620",
"projectId": "[PROJECT_ID]",
"region": "us-east5"
}
]
}

Autocomplete model

Anthropic Vertex AI currently does not offer any autocomplete models.

Click here to see a list of autocomplete model providers.

Embeddings model

Anthropic Vertex AI currently does not offer any embeddings models.

Click here to see a list of embeddings model providers.

Reranking model

Anthropic Vertex AI currently does not offer any reranking models.

Click here to see a list of reranking model providers.

Prompt caching

Anthropic Vertex AI supports prompt caching with Claude.

To enable caching of the system message and the turn-by-turn conversation, update your your model configuration as following:

{
"models": [
{
// Enable prompt caching
"cacheBehavior": {
"cacheSystemMessage": true,
"cacheConversation": true
},
"title": "Anthropic",
"provider": "anthropic",
"model": "claude-3-5-sonnet-20240620",
"apiKey": "[API_KEY]"
}
]
}