Virtual Kubelet的工作原理
从Kubernetes API服务器的角度来看,Virtual Kubelet看起来像普通的kubelet,但其关键区别在于它们在其他地方调度容器,例如在云无服务器API中,而不是在节点上。对内向k8s apiserver注册node,保持跟kubelet一样的协议通信,对外提供简化的pleg接口供第三方provider实现。常见的供应商有:
- Alibaba Cloud Elastic Container Instance (ECI)
- AWS Fargate
- Azure Batch
- Azure Container Instances (ACI)
- Kubernetes Container Runtime Interface (CRI)
- Huawei Cloud Container Instance (CCI)
- Hyper.sh
- HashiCorp Nomad
- Service Fabric Mesh
- vSphere Integrated Containers (VIC)
整体架构图:
// PodLifecycleHandler defines the interface used by the PodController to react // to new and changed pods scheduled to the node that is being managed. // // Errors produced by these methods should implement an interface from // github.com/virtual-kubelet/virtual-kubelet/errdefs package in order for the // core logic to be able to understand the type of failure. type PodLifecycleHandler interface { // CreatePod takes a Kubernetes Pod and deploys it within the provider. CreatePod(ctx context.Context, pod *corev1.Pod) error // UpdatePod takes a Kubernetes Pod and updates it within the provider. UpdatePod(ctx context.Context, pod *corev1.Pod) error // DeletePod takes a Kubernetes Pod and deletes it from the provider. DeletePod(ctx context.Context, pod *corev1.Pod) error // GetPod retrieves a pod by name from the provider (can be cached). // The Pod returned is expected to be immutable, and may be accessed // concurrently outside of the calling goroutine. Therefore it is recommended // to return a version after DeepCopy. GetPod(ctx context.Context, namespace, name string) (*corev1.Pod, error) // GetPodStatus retrieves the status of a pod by name from the provider. // The PodStatus returned is expected to be immutable, and may be accessed // concurrently outside of the calling goroutine. Therefore it is recommended // to return a version after DeepCopy. GetPodStatus(ctx context.Context, namespace, name string) (*corev1.PodStatus, error) // GetPods retrieves a list of all pods running on the provider (can be cached). // The Pods returned are expected to be immutable, and may be accessed // concurrently outside of the calling goroutine. Therefore it is recommended // to return a version after DeepCopy. GetPods(context.Context) ([]*corev1.Pod, error) }
通过这几个pleg接口设计可以看出:
1)对于增删改pod而言,是从控制面发起,由具体provider实现的逻辑。至于provider如何实现,也就对应于底层pod的具体不同实现。
2)而对于查pod而言,却是入参pod的相关信息,输出的是控制面pod的协议。查看pod的信息是通过k8s控制面输出的,k8s apiserver
会从etcd中获取pod的元数据,或者client端通过informer cached获取数据。翻看源码发现,provider提供了接口,但接口实现时机留
给业务方具体问题具体分析。常用的场景为provider端的pod的状态发生变化然后通过查询的接口同步变化到k8s控制面,至于控制面pod
变化同步到provider端,这个在增删改pod的时候会实现。
综上所述,简化的pleg接口实现了pod生命周期的维护,按需实现,能满足业务的基本诉求。而Virtual Kubelet的工作原理则是对内向apiserver
维持Kubelet的协议实现伪装,对外设计pleg以及exec、logs等接口供不同业务实现自定义的provider。