Early Preview

Ai-lib: Unified Multi-Provider AI API Client for Rust

One API client, multiple providers: reliable streaming, function calls, reasoning models, and observability — with <1ms overhead.

★ GitHub Quick Start Enterprise & Support

Dual-licensed MIT / Apache-2.0 · Production-grade abstractions · Vendor neutral

Core Value

Unified Providers

One abstraction spans OpenAI, Groq, Anthropic, Gemini, Mistral, Cohere, etc.

Consistent Streaming

Standard incremental deltas; hides raw SSE differences.

Reasoning Models

Built-in support for Groq Qwen, DeepSeek R1, and other reasoning models with structured output.

Reliability Primitives

Retries, timeout, error typing, proxy control, connection pooling.

Model Strategy

Performance / cost / health / weighted routing out of the box.

Low Overhead

Adds ~0.6–0.9ms per request (transparent methodology).

Progressive Complexity

Scale from quick_chat to custom transports & metrics gradually.

Install & Quick Start

Add the dependency, then use the unified API to call any supported provider immediately.

[dependencies]
ai-lib = "0.2.21"
tokio = { version = "1", features = ["full"] }
futures = "0.3"

Quick Start

use ai_lib::{AiClient, Provider, Message, Role, Content, ChatCompletionRequest};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = AiClient::new(Provider::Groq)?;
    let req = ChatCompletionRequest::new(
        client.default_chat_model(),
        vec![Message {
            role: Role::User,
            content: Content::Text("Hello, world!".to_string()),
            function_call: None,
        }]
    );
    let resp = client.chat_completion(req).await?;
    println!("Answer: {}", resp.choices[0].message.content.as_text());
    Ok(())
}

Environment Variables Configuration

# Set API keys
export GROQ_API_KEY=your_groq_api_key
export OPENAI_API_KEY=your_openai_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

# Proxy configuration (optional)
export AI_PROXY_URL=http://proxy.example.com:8080

Streaming

use futures::StreamExt;

let mut stream = client.chat_completion_stream(req).await?;
while let Some(chunk) = stream.next().await {
    let c = chunk?;
    if let Some(delta) = c.choices[0].delta.content.clone() {
        print!("{delta}");
    }
}

Supported Providers

A single Provider enum eliminates branching by vendor; mix local and cloud (e.g. Ollama + OpenAI) effortlessly.

OpenAI

Groq

Anthropic

Gemini

Mistral

Cohere

HuggingFace

TogetherAI

DeepSeek

Qwen

Baidu Wenxin

Tencent Hunyuan

iFlytek Spark

Moonshot Kimi

Azure OpenAI

Ollama

xAI Grok

Performance & Methodology

The following metrics refer to SDK layer overhead only (excludes remote model latency). See repository README for full methodology. Always benchmark with your workload.

0.6–0.9ms Per-request overhead

<2ms Streaming parse cost

11K–13K Mock throughput req/s

Note: Real-world throughput constrained by provider rate limits & network. Figures are indicative, not guarantees.

FAQ

Does it support local inference? Yes via Ollama; can mix with cloud providers.

How to know if an error is retryable? Errors are typed; use e.is_retryable().

Do you log request content? No. Content is not logged by default; add metrics hooks if needed.

Is function calling supported? Yes unified through Tool + FunctionCallPolicy.

When should I NOT use it? One-off small scripts targeting a single provider—use the vendor SDK directly.