LLM Providers¶

k13d supports multiple LLM providers for AI-powered features.

Supported Providers¶

Configuration¶

OpenAI¶

# ~/.config/k13d/config.yaml
llm:
  provider: openai
  model: gpt-4
  api_key: ${OPENAI_API_KEY}

Or via environment variable:

export OPENAI_API_KEY=sk-your-key-here
k13d -web

Anthropic (Claude)¶

llm:
  provider: anthropic
  model: claude-3-opus-20240229
  api_key: ${ANTHROPIC_API_KEY}

Google Gemini¶

llm:
  provider: gemini
  model: gemini-pro
  api_key: ${GOOGLE_API_KEY}

Ollama (Local)¶

Start Ollama:

ollama serve
ollama pull llama3.2

Configure k13d:

llm:
  provider: ollama
  model: llama3.2
  endpoint: http://localhost:11434

Azure OpenAI¶

llm:
  provider: azopenai
  model: gpt-4
  endpoint: https://your-resource.openai.azure.com/
  api_key: ${AZURE_OPENAI_API_KEY}
  api_version: "2024-02-15-preview"

AWS Bedrock¶

llm:
  provider: bedrock
  model: anthropic.claude-3-sonnet-20240229-v1:0
  # Uses AWS credentials from environment

Required environment variables:

export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
export AWS_REGION=us-east-1

Embedded LLM¶

Run without any external API:

k13d --embedded-llm -web

Configuration:

llm:
  provider: embedded
  model: llama-3.2-1b  # Built-in model

Multi-Model Configuration¶

Configure multiple models and switch between them:

models:
  - name: gpt-4
    provider: openai
    model: gpt-4
    api_key: ${OPENAI_API_KEY}

  - name: local-llama
    provider: ollama
    model: llama3.2
    endpoint: http://localhost:11434

  - name: claude
    provider: anthropic
    model: claude-3-opus-20240229
    api_key: ${ANTHROPIC_API_KEY}

# Default model
active_model: gpt-4

Switch models at runtime:

TUI:

:model local-llama

Web: Settings → AI → Active Model

Provider Features¶

Feature Comparison¶

Tool Calling Support¶

All providers support tool calling for kubectl and bash integration:

User: "Scale nginx to 5 replicas"
AI: [Calls kubectl scale deployment nginx --replicas=5]

Performance Considerations¶

Latency¶

*Depends on hardware

Cost Comparison¶

Recommended Configurations¶

For Best Quality¶

llm:
  provider: openai
  model: gpt-4-turbo

For Speed¶

llm:
  provider: openai
  model: gpt-3.5-turbo

For Privacy (Local)¶

llm:
  provider: ollama
  model: llama3.2
  endpoint: http://localhost:11434

For Air-Gapped¶

llm:
  provider: embedded
  model: llama-3.2-1b

Environment Variables¶

Troubleshooting¶

API Key Not Found¶

Error: OpenAI API key not found

Solution:

export OPENAI_API_KEY=sk-your-key
# or add to config.yaml

Rate Limit Exceeded¶

Error: Rate limit exceeded

Solutions: 1. Wait and retry 2. Upgrade API plan 3. Switch to different model

Ollama Connection Failed¶

Error: Connection refused localhost:11434

Solution:

# Start Ollama
ollama serve

# Verify it's running
curl http://localhost:11434/api/tags

Slow Responses¶

For faster responses: 1. Use GPT-3.5 instead of GPT-4 2. Use local Ollama with smaller model 3. Reduce context length

Security Best Practices¶

1. Use Environment Variables¶

# Good
export OPENAI_API_KEY=sk-...
k13d -web

# Bad - key in config file
llm:
  api_key: sk-actual-key-here

2. Rotate Keys Regularly¶

3. Use Least Privilege¶

Create API keys with minimal permissions.

4. Monitor Usage¶

Track API usage to detect anomalies.

Next Steps¶

• Embedded LLM - Run without API
• Tool Calling - How AI executes commands
• Benchmarks - Model performance comparison