原文作者意琦行本文转载自HAMi 正式接入 Kubernetes DRA下一代 GPU 资源模型实践指南HAMi 是目前 Kubernetes 上最活跃的开源 vGPU 方案能够将一块物理 GPU 按显存和算力细粒度地切分为多个虚拟 GPU供不同 Pod 共享。本文聚焦 HAMi DRA 模式的部署与使用安装 HAMi DRA 驱动后分别用原生模式和兼容模式提交 Pod验证 GPU 切分是否生效。Kubernetes 在 1.34 中正式 GA 了 DRADynamic Resource Allocation动态资源分配。DRA 的核心改进是让调度器参与资源分配在 Pod 调度阶段就精确匹配设备属性避免了 DevicePlugin “调度到节点后才发现资源不够” 的问题。HAMi 最近的版本已经正式接入了 DRA用户既可以使用原生 DRA 模式也可以用 DevicePlugin 兼容模式无缝迁移。什么是 HAMiHAMi异构 AI 计算虚拟化中间件是一个用于管理 Kubernetes 集群中异构 AI 计算设备的开源平台。前身为 k8s-vGPU-schedulerHAMi 可在多个容器和工作负载之间实现设备共享。HAMi 是云原生计算基金会CNCF的 Sandbox 项目并被收录于 CNCF 技术全景图和 CNAI 技术全景图。核心特性设备共享多设备支持兼容多种异构 AI 计算设备GPU、NPU 等共享访问多个容器可同时共享设备提高资源利用率内存管理硬限制在容器内强制执行严格的内存限制防止资源冲突动态分配根据工作负载需求按需分配设备内存灵活单位支持按 MB 或占总设备内存百分比的方式指定内存分配设备规格类型选择可请求特定类型的异构 AI 计算设备UUID 定向使用设备 UUID 精确指定特定设备易用性对工作负载透明容器内无需修改代码简单部署使用 Helm 轻松安装和卸载配置简单开放治理社区驱动由互联网、金融、制造业、云服务等多个领域的组织联合发起中立发展作为开源项目由 CNCF 管理HAMi 安装前提条件K8s 1.34 及以上版本同时开启 DRAConsumableCapacity Feature Gate1.34-1.35 DRAConsumableCapacity 默认未开启需要手动配置Container Runtime 必须开启 CDINVIDIA GPU 驱动 440 及以上版本特别是第一条DRAConsumableCapacity 在 1.36 才默认开启1.34、1.35 需手动配置。GPU Operator 安装安装 GPU Operator 时需要关闭 DevicePluginhelm repoaddnvidia https://helm.ngc.nvidia.com/nvidiahelm repo update helm upgrade--install--waitgpu-operator\-ngpu-operator --create-namespace\nvidia/gpu-operator\--versionv26.3.1\--setdriver.enabledtrue\--setdevicePlugin.enabledfalse--set devicePlugin.enabledfalse关闭 DevicePlugin避免与后续安装的 DRA Driver 冲突。安装 Cert-managerHAMi DRA Webhook 需要 TLS 证书因此需要提前安装 cert-manager 用于自动签发。helm repoaddcert-manager https://charts.jetstack.io helm repo update helminstallcert-manager cert-manager/cert-manager\-ncert-manager --create-namespace\--setcrds.enabledtrueHelm 安装 HAMi为节点打上gpuon标签未标记的节点不会被 HAMi 接管。#kubectl label nodes {nodeid} gpuonkubectl label nodes ecs-a10-shgpuon使用以下命令添加 HAMi 图表仓库helm repoaddhami-charts https://project-hami.github.io/HAMi/用以下命令进行安装# 核心是通过 --set dra.enabledtrue 开启 DRA 模式helm-nhami-systeminstallhami hami-charts/hami--setdra.enabledtrue --create-namespace注意DRA 模式与传统模式不兼容请勿同时启用。另外如果 GPU 驱动是主机预装非 GPU Operator 安装则安装时需额外指定--sethami-dra.drivers.nvidia.containerDriverfalse验证正常情况下会在 hami-system 下启动以下 PodrootECS-A10-SH:/data/nfs/shared-skills-cicd# k -n hami-system get poNAME READY STATUS RESTARTS AGE hami-dra-driver-kubelet-plugin-hflbh1/1 Running02m49s hami-hami-dra-monitor-7b484d5f95-rlkcg1/1 Running022m hami-hami-dra-webhook-64bfdc6b86-d4nlr1/1 Running022m使用查看 ResourceSlice查看 dra-driver 是否正常发布 resourceslicerootECS-A10-SH:/data/nfs/shared-skills-cicd# kubectl get resourcesliceNAME NODE DRIVER POOL AGE ecs-a10-sh-hami-core-gpu.project-hami.io-hnn6d ecs-a10-sh hami-core-gpu.project-hami.io ecs-a10-sh 119s详情如下rootECS-A10-SH:/data/nfs/shared-skills-cicd# kubectl get resourceslice ecs-a10-sh-hami-core-gpu.project-hami.io-hnn6d -oyamlapiVersion: resource.k8s.io/v1 kind: ResourceSlice metadata: creationTimestamp:2026-05-13T09:28:56ZgenerateName: ecs-a10-sh-hami-core-gpu.project-hami.io- generation:1name: ecs-a10-sh-hami-core-gpu.project-hami.io-hnn6d ownerReferences: - apiVersion: v1 controller:truekind: Node name: ecs-a10-sh uid: 76c7db94-fe0b-44ea-9b07-8bdb6132888b resourceVersion:61417761uid: 46d46b45-108e-45e3-98f2-000a091571d3 spec: devices: - attributes: architecture: string: Ampere brand: string: Nvidia cudaComputeCapability: version:8.6.0 cudaDriverVersion: version:12.4.0 driverVersion: version:550.144.3 minor: int:0pcieBusID: string: 0000:65:01.0 productName: string: NVIDIA A10 resource.kubernetes.io/pcieRoot: string: pci0000:64 type: string: hami-gpu uuid: string: GPU-f1c7d08c-ae21-13e7-0de0-9eb14ff71eaf capacity: cores: value:100memory: value: 23028Mi name: hami-gpu-0 driver: hami-core-gpu.project-hami.io nodeName: ecs-a10-sh pool: generation:1name: ecs-a10-sh resourceSliceCount:1可以看到ResourceSlice 记录了 GPU 的架构、型号、显存等信息。提交任务DRA 原生模式DRA 原生使用流程是先创建 ResourceClaim然后创建 Pod 使用该 ResourceClaim。提交任务ResourceClaim 以及对应 Pod 完整 yaml 如下# DRA 原生模式 - 手动创建 ResourceClaim# 申请 10G 显存 50 cores 的 A10 GPUapiVersion:resource.k8s.io/v1kind:ResourceClaimmetadata:name:gpu-half-claimspec:devices:requests:-name:gpuexactly:deviceClassName:hami-core-gpu.project-hami.ioallocationMode:ExactCountcount:1capacity:requests:cores:50memory:10Gi---apiVersion:v1kind:Podmetadata:name:gpu-test-dra-nativespec:containers:-name:cudaimage:172.31.0.2:5000/nvidia/cuda:13.0.1-base-ubi9command:[sleep,3600]resources:claims:-name:gpuresourceClaims:-name:gpuresourceClaimName:gpu-half-claimrestartPolicy:Never查看调度情况通过 ResourceClaim 可以看到资源分配情况rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k get poNAME READY STATUS RESTARTS AGE gpu-test-dra-native1/1 Running088s rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k get resourceclaimNAME STATE AGE gpu-half-claim allocated,reserved 21s rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k get resourceclaim gpu-half-claim -oyamlapiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: gpu-half-claim namespace: default spec: devices: requests: - exactly: allocationMode: ExactCount capacity: requests: cores:50memory: 10Gi count:1deviceClassName: hami-core-gpu.project-hami.io name: gpu status: allocation: devices: results: - consumedCapacity: cores:50memory: 10Gi device: hami-gpu-0 driver: hami-core-gpu.project-hami.io pool: ecs-a10-sh request: gpu shareID: 6108e68f-a7ec-4a30-9782-634885c0c728 nodeSelector: nodeSelectorTerms: - matchFields: - key: metadata.name operator: In values: - ecs-a10-sh reservedFor: - name: gpu-test-dra-native resource: pods uid: d99dc6df-092c-4f3a-ac55-cfb88c017af7效果Pod 中执行 nvidia-smi 看到显存是我们申请的 10G说明 HAMi 生效了。[HAMI-core Msg(51:140707774973760:libvgpu.c:870)]: Initializing..... Wed May1310:58:202026-----------------------------------------------------------------------------------------|NVIDIA-SMI550.144.03 Driver Version:550.144.03 CUDA Version:13.0||---------------------------------------------------------------------------------------|GPU Name Persistence-M|Bus-Id Disp.A|Volatile Uncorr. ECC||Fan Temp Perf Pwr:Usage/Cap|Memory-Usage|GPU-Util Compute M.||||MIG M.||||0NVIDIA A10 On|00000000:65:01.0 Off|0||0% 32C P8 22W / 150W|0MiB / 10240MiB|0% Default||||N/A|--------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------|Processes:||GPU GI CI PID Type Process name GPU Memory||ID ID Usage||||No running processes found|-----------------------------------------------------------------------------------------[HAMI-core Msg(51:140707774973760:multiprocess_memory_limit.c:703)]: Cleanup onexitforPID51[HAMI-core Msg(51:140707774973760:multiprocess_memory_limit.c:739)]: Exit cleanup completeforPID51提交任务DevicePlugin 兼容模式原生 DRA 模式需要手动创建 ResourceClaim对存量业务不够友好。为了便于大家迁移HAMi 提供了兼容模式用户仍然像 DevicePlugin 那样申请资源由 HAMi DRA Webhook自动拦截并转换为 ResourceClaim调度器分配后再挂载到 Pod。提交任务和使用 DevicePlugin 一样正常在 resources 中申请资源即可# 兼容模式 - 按传统方式申请 GPUHAMi webhook 自动转换为 ResourceClaim# 申请 1 块 GPU10Gi 显存 50% 算力apiVersion:v1kind:Podmetadata:name:gpu-test-compatiblespec:containers:-name:cudaimage:172.31.0.2:5000/nvidia/cuda:13.0.1-base-ubi9command:[sleep,3600]resources:limits:nvidia.com/gpu:1nvidia.com/gpumem:10240nvidia.com/gpucores:50restartPolicy:Never查看调度情况HAMi 会根据nvidia.com/gpumem、nvidia.com/gpucores自动生成 ResourceClaim并绑定到 Pod。对应的 ResourceClaim 如下rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k get resourceclaimNAME STATE AGE default-gpu-test-compatible-cuda allocated,reserved 2m47s rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k get resourceclaim default-gpu-test-compatible-cuda -oyamlapiVersion:resource.k8s.io/v1kind:ResourceClaimmetadata:creationTimestamp:2026-05-13T11:14:06Zfinalizers:-resource.kubernetes.io/delete-protectionname:default-gpu-test-compatible-cudanamespace:defaultresourceVersion:61451167uid:8212ef37-f71c-45ca-ac4a-f94ead923eefspec:devices:requests:-exactly:allocationMode:ExactCountcapacity:requests:cores:50memory:10737418240count:1deviceClassName:hami-core-gpu.project-hami.ioselectors:-cel:expression:device.attributes[hami-core-gpu.project-hami.io].type hami-gpuname:gpustatus:allocation:devices:results:-consumedCapacity:cores:50memory:10Gidevice:hami-gpu-0driver:hami-core-gpu.project-hami.iopool:ecs-a10-shrequest:gpushareID:a8dba99f-7841-41ad-9f07-5ec39ddee543nodeSelector:nodeSelectorTerms:-matchFields:-key:metadata.nameoperator:Invalues:-ecs-a10-shreservedFor:-name:gpu-test-compatibleresource:podsuid:173a6d7f-665b-4b2d-961c-f550d70f7484核心配置spec:devices:requests:-exactly:allocationMode:ExactCountcapacity:requests:cores:50memory:10737418240count:1对比原始 Pod 的资源申请resources:limits:nvidia.com/gpu:1nvidia.com/gpumem:10240nvidia.com/gpucores:50Webhook 转换映射关系原始资源申请ResourceClaim 字段nvidia.com/gpu: 1requests.count: 1nvidia.com/gpumem: 10240requests.capacity.memorynvidia.com/gpucores: 50requests.capacity.cores效果同样的显存为 10240M说明 HAMi 也生效了。rootECS-A10-SH:~/lixd/deploy/gpu/hami/examples# k exec -it gpu-test-compatible -- nvidia-smi[HAMI-core Msg(57:139707024262976:libvgpu.c:870)]: Initializing..... Wed May1311:21:392026-----------------------------------------------------------------------------------------|NVIDIA-SMI550.144.03 Driver Version:550.144.03 CUDA Version:13.0||---------------------------------------------------------------------------------------|GPU Name Persistence-M|Bus-Id Disp.A|Volatile Uncorr. ECC||Fan Temp Perf Pwr:Usage/Cap|Memory-Usage|GPU-Util Compute M.||||MIG M.||||0NVIDIA A10 On|00000000:65:01.0 Off|0||0% 32C P8 22W / 150W|0MiB / 10240MiB|0% Default||||N/A|--------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------|Processes:||GPU GI CI PID Type Process name GPU Memory||ID ID Usage||||No running processes found|-----------------------------------------------------------------------------------------[HAMI-core Msg(57:139707024262976:multiprocess_memory_limit.c:703)]: Cleanup onexitforPID57[HAMI-core Msg(57:139707024262976:multiprocess_memory_limit.c:739)]: Exit cleanup completeforPID57小结本文围绕 HAMi DRA 模式完成了从安装到验证的完整实践部署 HAMi DRA关闭 DevicePlugin 后通过 Helm 安装 HAMi开启dra.enabledtrueDRA 原生模式手动创建 ResourceClaim 声明显存与算力Pod 通过 resourceClaim 引用DevicePlugin 兼容模式沿用传统nvidia.com/gpu等资源申请HAMi DRA Webhook 自动转换为 ResourceClaim存量业务零改造即可迁移两种模式的核心差异在于 ResourceClaim 的创建方式——原生模式手动管理、兼容模式自动生成底层调度与切分逻辑一致。「密瓜智能Dynamia 」专注 GPU 虚拟化与异构算力调度发起并主导 CNCF 开源项目 HAMi同时基于 HAMi 提供商业发行版、企业产品与服务帮助用户在真实业务中规模化使用相关能力官网dynamia.ai邮箱infodynamia.aiHAMi 项目地址https://github.com/project-hami/hami本文作者「Dynamia密瓜智能」————————————————版权声明本文为CSDN博主「密瓜智能」的原创文章遵循CC 4.0 BY-SA版权协议转载请附上原文出处链接及本声明。原文链接https://www.lixueduan.com/posts/kubernetes/56-hami-dra-quickstart/