OX
OnglX

API Components

Deploy OpenAI-compatible inference APIs

OpenAI Compatible
AWS Bedrock
Auto-scaling

Overview

API components provide OpenAI-compatible inference endpoints that connect to AWS Bedrock models. This allows you to use existing OpenAI client libraries and tools while leveraging AWS Bedrock's powerful foundation models.

The API automatically handles authentication, model routing, request formatting, and response translation between OpenAI and AWS Bedrock formats.

Adding an API Component

Add an API component to your deployment:

Terminal - OnglX Deploy
$ onglx-deploy add api
✓ Added api component with OpenAI compatibility
✓ Configured AWS Bedrock access
✓ Set up automatic scaling
Component details:
  Type: api
  Runtime: AWS Lambda
  Models: All available Bedrock models
  Authentication: API key based

Features

OpenAI Compatibility
  • • Drop-in replacement for OpenAI API
  • • Same request/response format
  • • Compatible with existing libraries
  • • Streaming and non-streaming support
AWS Integration
  • • Direct AWS Bedrock access
  • • No third-party dependencies
  • • AWS-native authentication
  • • CloudWatch logging

Usage Examples

Using with OpenAI Python SDK

1import openai
2
3# Configure client with your API endpoint
4client = openai.OpenAI(
5    api_key="your-api-key",
6    base_url="https://your-api-endpoint.amazonaws.com"
7)
8
9# Use Claude 3 via OpenAI interface
10response = client.chat.completions.create(
11    model="anthropic.claude-3-sonnet-20240229-v1:0",
12    messages=[
13        {"role": "user", "content": "Hello, how are you?"}
14    ]
15)
16
17print(response.choices[0].message.content)

Direct HTTP Requests

1curl -X POST https://your-api-endpoint.amazonaws.com/v1/chat/completions \
2  -H "Content-Type: application/json" \
3  -H "Authorization: Bearer your-api-key" \
4  -d '{
5    "model": "anthropic.claude-3-sonnet-20240229-v1:0",
6    "messages": [
7      {"role": "user", "content": "What is AWS Bedrock?"}
8    ]
9  }'

Streaming Responses

1import openai
2
3client = openai.OpenAI(
4    api_key="your-api-key",
5    base_url="https://your-api-endpoint.amazonaws.com"
6)
7
8# Stream response for real-time UX
9stream = client.chat.completions.create(
10    model="anthropic.claude-3-sonnet-20240229-v1:0",
11    messages=[{"role": "user", "content": "Write a story"}],
12    stream=True
13)
14
15for chunk in stream:
16    if chunk.choices[0].delta.content:
17        print(chunk.choices[0].delta.content, end="")

Supported Models

Your API component supports all AWS Bedrock foundation models:

Text Generation
  • • anthropic.claude-3-sonnet-20240229-v1:0
  • • anthropic.claude-3-haiku-20240307-v1:0
  • • anthropic.claude-3-opus-20240229-v1:0
  • • amazon.titan-text-express-v1
  • • meta.llama2-70b-chat-v1
Embeddings
  • • amazon.titan-embed-text-v1
  • • cohere.embed-english-v3
  • • cohere.embed-multilingual-v3

🚀 Next Steps

  • • Deploy your API: onglx-deploy deploy
  • • Check API status: onglx-deploy status
  • • View API logs: onglx-deploy logs api
  • • Add a UI component