agentic_huge_data_base / wiki
页面 Onyx · 2.2 Kubernetes 部署·DeepWiki 中文全文译文

2.2 · Kubernetes 部署(Kubernetes Deployment)

企业连接器与统一搜索 · 聚焦本章的模块关系、源码依据与实现要点。

项目Onyx 章节2.2 状态全文译文 模块存储与持久化、系统架构、配置治理、检索、召回与索引
源码线索
  • ct.yaml
  • deployment/helm/charts/onyx/Chart.lock
  • deployment/helm/charts/onyx/Chart.yaml
  • deployment/helm/charts/onyx/templates/api-deployment.yaml
  • deployment/helm/charts/onyx/templates/celery-beat.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-docfetching.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-docprocessing.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-heavy.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-light.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-monitoring.yaml
模块标签
  • 存储与持久化
  • 系统架构
  • 配置治理
  • 检索、召回与索引
  • 安装与启动

章节正文

Kubernetes 部署

Kubernetes 部署

相关源文件

本章引用的主要源码文件:

  • ct.yaml
  • deployment/helm/charts/onyx/Chart.lock
  • deployment/helm/charts/onyx/Chart.yaml
  • deployment/helm/charts/onyx/templates/api-deployment.yaml
  • deployment/helm/charts/onyx/templates/celery-beat.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-docfetching.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-docprocessing.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-heavy.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-light.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-monitoring.yaml
  • deployment/helm/charts/onyx/templates/celery-worker-primary.yaml
  • deployment/helm/charts/onyx/templates/configmap.yaml
  • deployment/helm/charts/onyx/templates/indexing-model-deployment.yaml
  • deployment/helm/charts/onyx/templates/inference-model-deployment.yaml
  • deployment/helm/charts/onyx/templates/webserver-deployment.yaml
  • deployment/helm/charts/onyx/values.yaml
  • deployment/terraform/modules/aws/README.md
  • deployment/terraform/modules/aws/eks/main.tf
  • deployment/terraform/modules/aws/eks/outputs.tf
  • deployment/terraform/modules/aws/eks/variables.tf
  • deployment/terraform/modules/aws/onyx/main.tf
  • deployment/terraform/modules/aws/onyx/outputs.tf
  • deployment/terraform/modules/aws/onyx/variables.tf
  • deployment/terraform/modules/aws/onyx/versions.tf
  • deployment/terraform/modules/aws/opensearch/main.tf
  • deployment/terraform/modules/aws/opensearch/outputs.tf
  • deployment/terraform/modules/aws/opensearch/variables.tf
  • deployment/terraform/modules/aws/postgres/main.tf
  • deployment/terraform/modules/aws/postgres/outputs.tf
  • deployment/terraform/modules/aws/postgres/variables.tf
  • deployment/terraform/modules/aws/s3/main.tf
  • deployment/terraform/modules/aws/s3/variables.tf
  • deployment/terraform/modules/aws/vpc/main.tf
  • deployment/terraform/modules/aws/vpc/outputs.tf
  • deployment/terraform/modules/aws/waf/main.tf
  • deployment/terraform/modules/aws/waf/outputs.tf
  • deployment/terraform/modules/aws/waf/variables.tf

本文档记录了 Onyx 的 Kubernetes 部署架构。内容涵盖使用 Helm Chart 实现的系统部署,包括所有核心组件的 Deployment、StatefulSet、Service、ConfigMap 和持久化卷的配置。

有关 Docker Compose 部署配置,请参见 Docker Compose 部署。有关环境变量配置的详细信息,请参见 环境配置。有关 Kubernetes 环境中的数据库模式管理和迁移,请参见 数据库迁移

部署架构总览

Onyx 的 Kubernetes 部署主要通过位于 deployment/helm/charts/onyx 的综合性 Helm Chart 进行管理 deployment/helm/charts/onyx/Chart.yaml:1-50。该架构采用微服务模式,将无状态应用逻辑与有状态数据存储分离。

核心组件映射
Docker Compose 服务Kubernetes 实体Helm 模板/子 Chart镜像
api_serverDeploymentapi-deployment.yamlonyxdotapp/onyx-backend
web_serverDeploymentwebserver-deployment.yamlonyxdotapp/onyx-web-server
relational_db集群(CNPG)postgresql 子 Chartcloudnative-pg 操作器
indexStatefulSetvespa 子 Chartvespaengine/vespa
cacheDeploymentredis 子 Chartredis
inference_modelDeploymentinference-model-deployment.yamlonyxdotapp/onyx-model-server
indexing_modelDeploymentindexing-model-deployment.yamlonyxdotapp/onyx-model-server
系统数据流

下图将高层系统名称与 Helm 模板中定义的特定 Kubernetes 服务和部署实体关联起来。

Onyx · 系统数据流 · 图 1
Onyx · 系统数据流 · 图 1

来源:deployment/helm/charts/onyx/Chart.yaml:1-50deployment/helm/charts/onyx/values.yaml:1-239deployment/helm/charts/onyx/templates/api-deployment.yaml:1-111

Helm Chart 配置

Onyx 的 Helm Chart 使用 values.yaml 文件来驱动所有子 Chart 和本地模板的配置。

全局设置

Chart 定义了镜像版本和主机配置的全局变量,以确保整个技术栈的一致性 deployment/helm/charts/onyx/values.yaml:5-11

组件开关

主开关 vectorDB.enabled(默认值:true)控制索引基础设施的部署 deployment/helm/charts/onyx/values.yaml:36-37。当禁用时:

  • 后端 Pod 上会设置 DISABLE_VECTOR_DBtrue
  • 索引模型服务器和所有 Celery 工作器部署(primary、light、heavy、docfetching 等)会被跳过 deployment/helm/charts/onyx/values.yaml:28-35
API 服务器初始化

API 服务器部署在启动过程中会自动处理数据库迁移,在启动 uvicorn 进程之前执行 alembic upgrade head deployment/helm/charts/onyx/templates/api-deployment.yaml:73-75

来源:deployment/helm/charts/onyx/values.yaml:1-239deployment/helm/charts/onyx/templates/api-deployment.yaml:1-111

后台工作器专业化

Kubernetes 部署将后台工作拆分为专门的工作器池,以实现独立的扩缩容和资源分配。

工作器池定义

每种工作器类型都在各自的模板文件中定义,并监听特定的 Celery 队列。

工作器部署队列/角色配置要点
lightvespa_metadata_syncconnector_deletiondoc_permissions_upsertopensearch_migration处理元数据同步和清理任务 deployment/helm/charts/onyx/templates/celery-worker-light.yaml:70-72
heavyconnector_pruningconnector_doc_permissions_synccsv_generationsandbox处理资源密集型的修剪和权限同步 deployment/helm/charts/onyx/templates/celery-worker-heavy.yaml:64-72
primaryceleryperiodic_tasks通用任务处理和 Celery Beat 任务消费 deployment/helm/charts/onyx/templates/celery-worker-primary.yaml:64-72
monitoringmonitoring专门用于系统健康监控任务的池 deployment/helm/charts/onyx/templates/celery-worker-monitoring.yaml:64-72
beat不适用触发周期性任务的调度器 deployment/helm/charts/onyx/templates/celery-beat.yaml:62-67
健康探针

工作器使用专门的 Python 探针脚本 onyx/background/celery/celery_k8s_probe.py 进行就绪探针和存活探针检查,以确保 Celery 消费者实际可用 deployment/helm/charts/onyx/templates/celery-worker-light.yaml:100-112

Onyx · 健康探针 · 图 2
Onyx · 健康探针 · 图 2

来源:deployment/helm/charts/onyx/templates/celery-worker-light.yaml:1-118deployment/helm/charts/onyx/templates/celery-worker-heavy.yaml:1-118deployment/helm/charts/onyx/templates/celery-worker-primary.yaml:1-118deployment/helm/charts/onyx/templates/celery-worker-monitoring.yaml:1-118

AI 模型服务器

Onyx 区分了推理(实时聊天/搜索)和索引(批量文档处理)模型服务器。

推理模型服务器
  • 用途: 为用户查询和聊天交互提供嵌入向量服务。
  • 服务: 通过 inference-model-service 在端口 9000 上暴露 deployment/helm/charts/onyx/values.yaml:141-147
  • 资源: 默认请求 2 个 CPU 和 3Gi 内存 deployment/helm/charts/onyx/values.yaml:162-164
索引模型服务器
  • 用途: 批量处理文档以构建向量索引。
  • 配置: 设置 INDEXING_ONLY="True" 以优化批量吞吐量 deployment/helm/charts/onyx/templates/indexing-model-deployment.yaml:68-69
  • 优化: 启动命令中包含 --limit-concurrency 标志(默认值:10),以防止在大量索引期间发生内存溢出(OOM)deployment/helm/charts/onyx/templates/indexing-model-deployment.yaml:59

来源:deployment/helm/charts/onyx/values.yaml:140-221deployment/helm/charts/onyx/templates/indexing-model-deployment.yaml:1-95

数据层与持久化

PostgreSQL(CloudNativePG)

Onyx 使用 CloudNativePG 操作器来管理 PostgreSQL。

  • 名称覆盖: 必须设置为 cloudnative-pg 以便操作器发现 deployment/helm/charts/onyx/values.yaml:15-18
  • 存储: 默认值为 10Gi,并启用 enableSuperuserAccess: true deployment/helm/charts/onyx/values.yaml:19-24
Vespa
  • 存储: 部署时使用 30Gi 的 PersistentVolumeClaim 用于索引 deployment/helm/charts/onyx/values.yaml:43-51
  • 安全: 以 root 身份运行并具有特权上下文,以允许 Vespa 的内部内存管理 deployment/helm/charts/onyx/values.yaml:63-65
OpenSearch
  • 初始化: 对于 2.12 及以上版本,需要设置 OPENSEARCH_INITIAL_ADMIN_PASSWORD deployment/helm/charts/onyx/values.yaml:99-103
  • Java 选项: 堆大小通过 opensearchJavaOpts 自动配置(默认值:4g)deployment/helm/charts/onyx/values.yaml:122-125

来源:deployment/helm/charts/onyx/values.yaml:13-126deployment/helm/charts/onyx/Chart.yaml:20-50

基础设施即代码(Terraform)

对于 AWS 部署,Onyx 提供了 Terraform 模块来配置底层的 EKS 集群和托管服务。

EKS 集群配置

eks 模块负责创建 Kubernetes 控制平面和节点组,包括用于 S3 和 RDS 访问的 IRSA(服务账户的 IAM 角色)deployment/terraform/modules/aws/onyx/main.tf:69-94

托管数据存储

生产环境通常使用托管的 AWS 服务,而不是集群内部署:

  • RDS PostgreSQL: 通过 postgres 模块配置,包含自动备份和 CloudWatch 告警 deployment/terraform/modules/aws/onyx/main.tf:46-60
  • Elasticache Redis: 配置了 auth_token 以实现传输加密 deployment/terraform/modules/aws/onyx/main.tf:33-44
  • OpenSearch 服务: 支持多可用区部署,包含备用节点和专用主节点 deployment/terraform/modules/aws/onyx/main.tf:112-145
Onyx · 托管数据存储 · 图 3
Onyx · 托管数据存储 · 图 3

来源:deployment/terraform/modules/aws/onyx/main.tf:1-145deployment/terraform/modules/aws/onyx/variables.tf:1-256deployment/terraform/modules/aws/postgres/main.tf:1-60