LLM API 网关 · 企业私有化部署

基于快手万擎网关 · 多租户隔离 · 账户额度管控 · 两种方案可选

方案 A：New API（一栈式企业网关）

适合场景：需要 Web 管理后台、多租户 Key 管理、开箱即用的额度/限流/计费系统。

QuantumNous/new-api ⭐ 35,956

架构

账户隔离

万擎渠道配置

# New API 渠道配置
channels:
  - name: "快手万擎-DeepSeek"
    type: "openai"
    base_url: "https://wanqing.kuaishou.com/v1"
    key: "${WANQING_API_KEY}"
    models: ["deepseek-chat", "deepseek-reasoner"]
    groups: ["研发部", "产品部"]   # 限定可用部门

  - name: "快手万擎-混元"
    type: "openai"
    base_url: "https://wanqing.kuaishou.com/v1"
    key: "${WANQING_API_KEY}"
    models: ["hunyuan-turbo", "hunyuan-pro"]

  - name: "Anthropic-Claude"
    type: "anthropic"
    base_url: "https://api.anthropic.com"
    key: "${ANTHROPIC_API_KEY}"
    models: ["claude-opus-4-8", "claude-sonnet-4-6"]

部署

# docker-compose.yml
services:
  postgres:
    image: postgres:16

  redis:
    image: redis:7-alpine

  new-api:
    image: calciumion/new-api:latest
    ports: ["3000:3000"]
    environment:
      SQL_DSN: "postgres://admin:${DB_PASSWORD}@postgres/apigateway"
      REDIS_CONN_STRING: "redis://redis:6379"
      SESSION_SECRET: ${SESSION_SECRET}
    depends_on: [postgres, redis]

优点 / 缺点

方案 B：LiteLLM（轻量模型代理）

适合场景：已有自建用户体系，只需统一模型接入 + 负载均衡 + cost tracking + fallback。

BerriAI/litellm ⭐ 23,000+

架构

账户隔离（LiteLLM Virtual Key + Team）

万擎代理配置

# LiteLLM proxy config.yaml
model_list:
  - model_name: deepseek-chat
    litellm_params:
      model: openai/deepseek-chat
      api_base: https://wanqing.kuaishou.com/v1
      api_key: ${WANQING_API_KEY}
      rpm: 1000
      tpm: 5000000

  - model_name: hunyuan-turbo
    litellm_params:
      model: openai/hunyuan-turbo
      api_base: https://wanqing.kuaishou.com/v1
      api_key: ${WANQING_API_KEY}

  - model_name: claude-opus-4-8
    litellm_params:
      model: anthropic/claude-opus-4-8
      api_key: ${ANTHROPIC_API_KEY}

litellm_settings:
  fallbacks:
    - deepseek-chat: qwen-max     # 万擎挂了自动切备用
  num_retries: 3
  request_timeout: 600

general_settings:
  master_key: ${LITELLM_MASTER_KEY}
  database_url: "postgresql://..."

router_settings:
  allowed_fails: 3
  cooldown_time: 30               # 冷却 30 秒再重试

Virtual Key + Team 配置示例

# 创建 Team + 分配额度
curl -X POST http://localhost:4000/team/new \
  -H "Authorization: Bearer ${MASTER_KEY}" \
  -d '{"team_alias":"rd-dept","models":["deepseek-chat","hunyuan-turbo"],"max_budget":100,"tpm_limit":500000}'

# 给小王发 Key
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer ${MASTER_KEY}" \
  -d '{"team_id":"rd-dept","key_alias":"wang-key","max_budget":30,"rpm_limit":20}'

部署

# docker-compose.yml
services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports: ["4000:4000"]
    volumes:
      - ./litellm_config.yaml:/app/config.yaml
    environment:
      DATABASE_URL: "postgresql://user:pass@postgres/litellm"
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
      STORE_MODEL_IN_DB: "true"

  postgres:
    image: postgres:16

优点 / 缺点

选型对比