k8s 编写 CRD subresources

在写 k8s的CRD controller的时候遇到需要用到subresources的情况

首先编写CRD文件的时候生命subresource

这里用kube-batch举例子

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: podgroups.scheduling.sigs.dev
spec:
  group: scheduling.sigs.dev
  names:
    kind: PodGroup
    plural: podgroups
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            minMember:
              format: int32
              type: integer
            queue:
              type: string
            priorityClassName:
              type: string
          type: object
        status:
          properties:
            succeeded:
              format: int32
              type: integer
            failed:
              format: int32
              type: integer
            running:
              format: int32
              type: integer
          type: object
      type: object
  version: v1alpha2
  subresources:
    status: {}

 

所使用的types.go

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// PodGroup is a collection of Pod; used for batch workload.
type PodGroup struct {
	metav1.TypeMeta `json:",inline"`
	// Standard object's metadata.
	// More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata
	// +optional
	metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`

	// Specification of the desired behavior of the pod group.
	// More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status
	// +optional
	Spec PodGroupSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`

	// Status represents the current information about a pod group.
	// This data may not be up to date.
	// +optional
	Status PodGroupStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}

// PodGroupSpec represents the template of a pod group.
type PodGroupSpec struct {
	// MinMember defines the minimal number of members/tasks to run the pod group;
	// if there's not enough resources to start all tasks, the scheduler
	// will not start anyone.
	MinMember int32 `json:"minMember,omitempty" protobuf:"bytes,1,opt,name=minMember"`

	// Queue defines the queue to allocate resource for PodGroup; if queue does not exist,
	// the PodGroup will not be scheduled.
	Queue string `json:"queue,omitempty" protobuf:"bytes,2,opt,name=queue"`

	// If specified, indicates the PodGroup's priority. "system-node-critical" and
	// "system-cluster-critical" are two special keywords which indicate the
	// highest priorities with the former being the highest priority. Any other
	// name must be defined by creating a PriorityClass object with that name.
	// If not specified, the PodGroup priority will be default or zero if there is no
	// default.
	// +optional
	PriorityClassName string `json:"priorityClassName,omitempty" protobuf:"bytes,3,opt,name=priorityClassName"`
}

// PodGroupStatus represents the current state of a pod group.
type PodGroupStatus struct {
	// Current phase of PodGroup.
	Phase PodGroupPhase `json:"phase,omitempty" protobuf:"bytes,1,opt,name=phase"`

	// The conditions of PodGroup.
	// +optional
	Conditions []PodGroupCondition `json:"conditions,omitempty" protobuf:"bytes,2,opt,name=conditions"`

	// The number of actively running pods.
	// +optional
	Running int32 `json:"running,omitempty" protobuf:"bytes,3,opt,name=running"`

	// The number of pods which reached phase Succeeded.
	// +optional
	Succeeded int32 `json:"succeeded,omitempty" protobuf:"bytes,4,opt,name=succeeded"`

	// The number of pods which reached phase Failed.
	// +optional
	Failed int32 `json:"failed,omitempty" protobuf:"bytes,5,opt,name=failed"`
}

　

注意的几个地方。

1. 当用yaml文件提交的时候，设置了status属性是没用的，因为默认提交的地址是资源的路径 也就是 /apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}

所以这个subresources的状态设置不上去

例如我提交一个

apiVersion: scheduling.incubator.k8s.io/v1alpha1
kind: PodGroup
metadata:
  name: qj-1
spec:
  minMember: 6
# 这个属性不会被kubectl命令设置上去
status
  succeeded: 1

　

来看下 CRD subresouces在k8s中的定义

// CustomResourceSubresources defines the status and scale subresources for CustomResources.
type CustomResourceSubresources struct {
	// Status denotes the status subresource for CustomResources
	Status *CustomResourceSubresourceStatus
	// Scale denotes the scale subresource for CustomResources
	Scale *CustomResourceSubresourceScale
}

// CustomResourceSubresourceStatus defines how to serve the status subresource for CustomResources.
// Status is represented by the `.status` JSON path inside of a CustomResource. When set,
// * exposes a /status subresource for the custom resource
// * PUT requests to the /status subresource take a custom resource object, and ignore changes to anything except the status stanza
// * PUT/POST/PATCH requests to the custom resource ignore changes to the status stanza
type CustomResourceSubresourceStatus struct{}

// CustomResourceSubresourceScale defines how to serve the scale subresource for CustomResources.
type CustomResourceSubresourceScale struct {
   // SpecReplicasPath defines the JSON path inside of a CustomResource that corresponds to Scale.Spec.Replicas.
   // Only JSON paths without the array notation are allowed.
   // Must be a JSON Path under .spec.
   // If there is no value under the given path in the CustomResource, the /scale subresource will return an error on GET.
   SpecReplicasPath string
   // StatusReplicasPath defines the JSON path inside of a CustomResource that corresponds to Scale.Status.Replicas.
   // Only JSON paths without the array notation are allowed.
   // Must be a JSON Path under .status.
   // If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource
   // will default to 0.
   StatusReplicasPath string
   // LabelSelectorPath defines the JSON path inside of a CustomResource that corresponds to Scale.Status.Selector.
   // Only JSON paths without the array notation are allowed.
   // Must be a JSON Path under .status or .spec.
   // Must be set to work with HPA.
   // The field pointed by this JSON path must be a string field (not a complex selector struct)
   // which contains a serialized label selector in string form.
   // More info: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions#scale-subresource
   // If there is no value under the given path in the CustomResource, the status label selector value in the /scale
   // subresource will default to the empty string.
   // +optional
   LabelSelectorPath *string
}

 

支持 Status, Scale 两种,

// * PUT requests to the /status subresource take a custom resource object, and ignore changes to anything except the status stanza
// * PUT/POST/PATCH requests to the custom resource ignore changes to the status stanza

当请求为 /status 的时会忽略对象的其他字段只修改 status

也就是说：如果想用命令行方式修改status字段，需要提交到 /apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}/status

 

2. 在测试的时候如果想手动修改状态怎么办

要修改状态必须先获取当前对象，因为有reversion的限制，提交的对象会进行严重，所以先get

随后在get出来的对象中加入status字段

curl --cacert {ca.pen} -H "Authorization: Bearer {token}"  APISERVER/apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}

假如返回结果为

{

...

"spec":{...}

}

修改添加status

{

...

"spec":{...},

"status":{"success":1}

}

随后用这个body去请求

 

curl --cacert {ca.pen} -H "Authorization: Bearer {token}"   -H "Content-Type: application/json" --request PUT APISERVER/apis/{groupname}/{version}/namespaces/{ns}/{resourcesname}/{name}/status --data '上边body'

 

记得一定要先去请求最新的对象，因为会验证resourceVersion