Typhoon API Documentation

Quickstart

curl --location 'https://api.opentyphoon.ai/v1/chat/completions' \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <*** OPEN TYPHOON API KEY ***>' \
    --data '{
        "model": "typhoon-v2-70b-instruct",
        "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant. You must answer only in Thai."
        },
        {
            "role": "user",
            "content": "ขอสูตรไก่ย่าง"
        }
        ],
        "max_tokens": 512,
        "temperature": 0.6,
        "top_p": 0.95,
        "repetition_penalty": 1.05,
        "stream": false
    }'

API Reference

The Typhoon API is an OpenAI compatible API with the following parameters:

• model (str, required): “typhoon-instruct”
• messages (array, required): {"role": "user" | "system" | "assistant", "content": "<_ CONTENT _>"}
• max_tokens (int): Maximum number of tokens to generate. Requests can use up to 8192 tokens shared between prompt and completion. (default: 150)
• temperature (float): Sampling temperature. Higher values means the result will be more random, lower values means the result will be more deterministic. We recommend a temperature less than 0.6.
• top_p (float): An alternative to sampling with temperature.
• stream (bool): If true, the API will return partial responses in real time as they are generated. The default is false.
• repetition_penalty (float): The parameter for repetition penalty, much like in the transformers library. We suggest repetition_penalty = 1.05 if you get a repetitive response.

Model list

Model	Size	Rate Limit	Release Date	End of Life
typhoon-v1.5-instruct	8B	20req/s 150req/m	09-05-24	-
typhoon-v1.5x-70b-instruct	70B	5req/s 50req/m	01-06-24	-
typhoon-v2-8b-instruct	8B	5req/s 50req/m	19-12-24	-
typhoon-v2-70b-instruct	70B	5req/s 50req/m	19-12-24	-

Examples

OpenAI integration

You can use the OpenAI client to connect with Typhoon by specifying base_url='https://api.opentyphoon.ai/v1' and api_key=<*** OPEN TYPHOON API KEY ***>.

from openai import OpenAI
client = OpenAI(
   api_key='<*** OPEN TYPHOON API KEY ***>',
   base_url='https://api.opentyphoon.ai/v1'
)

chat_completion = client.chat.completions.create(
    model="typhoon-v2-70b-instruct",
    messages=[{"role": "user", "content": "สวัสดี"}]
)
print(chat_completion.choices[0].message.content)

Llama-index integration

You can use the OpenAILike package of LlamaIndex to integrate with LlamaIndex.

from llama_index.core.llms import ChatMessage, MessageRole
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(model="typhoon-v2-70b-instruct",
                api_base="https://api.opentyphoon.ai/v1",
                context_window=32768,
                is_chat_model=True,
                max_tokens=1024,
                is_function_calling_model=False,
                api_key="<** API KEY ***>")

response = llm.chat([ChatMessage(role=MessageRole.USER, content='สวัสดี')])
print(str(response))

Langchain integration

You can use the ChatOpenAI interface to interact with Typhoon by supplying the base_url and api_key parameters from OpenTyphoon.ai.

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

client = ChatOpenAI(base_url='https://api.opentyphoon.ai/v1',
                    model='typhoon-instruct',
                    api_key="<** API KEY ***>")
resp = client.invoke([HumanMessage(content="สวัสดี")])
print(resp.content)

Stream support

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("OPENTYPHOON_API_KEY"),
    base_url="https://api.opentyphoon.ai/v1",
)

stream = client.chat.completions.create(
    model="typhoon-v2-70b-instruct",
    messages=[
        {
            "role": "user",
            "content": "ขอสูตรไก่ย่างหน่อย",
        }
    ],
    max_tokens=512,
    temperature=0.6,
    top_p=0.95,
    repetition_penalty=1.05,
    stream=True,
)

for chunk in stream:
    print(chunk)