Anthropic Vertex AI
info
You need to enable the Vertex AI API and set up the Google Application Default Credentials.
Chat model
We recommend configuring Claude 3.5 Sonnet as your chat model.
config.json
{
"models": [
{
"title": "Claude 3.5 Sonnet",
"provider": "anthropic-vertexai",
"model": "claude-3-5-sonnet-20240620",
"projectId": "[PROJECT_ID]",
"region": "us-east5"
}
]
}
Autocomplete model
Anthropic Vertex AI currently does not offer any autocomplete models.
Click here to see a list of autocomplete model providers.
Embeddings model
Anthropic Vertex AI currently does not offer any embeddings models.
Click here to see a list of embeddings model providers.
Reranking model
Anthropic Vertex AI currently does not offer any reranking models.
Click here to see a list of reranking model providers.
Prompt caching
Anthropic Vertex AI supports prompt caching with Claude.
To enable caching of the system message and the turn-by-turn conversation, update your your model configuration as following:
{
"models": [
{
// Enable prompt caching
"cacheBehavior": {
"cacheSystemMessage": true,
"cacheConversation": true
},
"title": "Anthropic",
"provider": "anthropic",
"model": "claude-3-5-sonnet-20240620",
"apiKey": "[API_KEY]"
}
]
}