k8s health check probes

在 Kubernetes 中，`readiness`、`startup` 和 `liveness` 探针可以同时定义，并且它们的生命周期和作用是不同的。以下是它们的详细区别和生命周期：

---

### 1. **Readiness Probe**
- **作用**: 用于判断容器是否已经准备好接收流量。
- **生命周期**: 
  - 当容器启动后，Kubernetes 会定期执行 `readiness` 探针。
  - 如果探针失败，容器会被标记为 "Not Ready"，并且不会接收流量（例如，Service 不会将流量路由到该容器）。
  - 如果探针成功，容器会被标记为 "Ready"，并开始接收流量。
- **典型场景**:
  - 应用需要一些初始化操作（例如加载配置文件或建立数据库连接）才能正常工作。
  - 确保流量只发送到已经准备好的容器。

---

### 2. **Startup Probe**
- **作用**: 用于判断容器是否已经成功启动。
- **生命周期**:
  - `startup` 探针主要用于检测容器的启动过程是否完成。
  - 在 `startup` 探针定义的情况下，`liveness` 探针会被延迟执行，直到 `startup` 探针成功。
  - 如果 `startup` 探针失败，容器会被认为启动失败，并被重启。
  - 一旦 `startup` 探针成功，Kubernetes 不会再执行它。
- **典型场景**:
  - 应用启动时间较长，可能需要执行复杂的初始化操作。
  - 避免在启动过程中触发 `liveness` 探针导致容器被错误地重启。

---

### 3. **Liveness Probe**
- **作用**: 用于判断容器是否仍然处于健康状态。
- **生命周期**:
  - 当容器启动后，Kubernetes 会定期执行 `liveness` 探针。
  - 如果探针失败，容器会被认为已经崩溃或进入不可恢复的状态，并被重启。
- **典型场景**:
  - 应用可能会因为内部错误进入不可恢复的状态。
  - 确保容器在出现问题时能够自动恢复。

---

### 生命周期的关系
- 如果定义了 `startup` 探针，`liveness` 探针会被延迟执行，直到 `startup` 探针成功。
- `readiness` 探针和 `startup` 探针可以同时定义，但它们的作用不同：
  - `startup` 探针用于判断容器是否启动完成。
  - `readiness` 探针用于判断容器是否准备好接收流量。
- 如果没有定义 `startup` 探针，`liveness` 探针会在容器启动后立即开始执行。

---

### 其他区别
| **探针类型** | **失败后行为** | **成功后行为** | **主要用途** |
|--------------|----------------|----------------|--------------|
| `readiness`  | 容器被标记为 "Not Ready"，停止接收流量 | 容器被标记为 "Ready"，开始接收流量 | 控制流量路由 |
| `startup`    | 容器被重启 | 探针停止执行 | 确保容器启动完成 |
| `liveness`   | 容器被重启 | 探针继续执行 | 检测容器健康状态 |

 

Kubernetes uses health checks to ensure the availability and reliability of applications running within containers. These health checks continuously monitor the health status of containers and allow Kubernetes to take appropriate actions if an application becomes unresponsive or enters an unhealthy state.

The three types of health checks supported by Kubernetes are:

1. Liveness Probes

Liveness probes are responsible for determining whether a container is alive and running as expected. They help detect scenarios where an application is running, but it may not be able to handle requests correctly due to internal issues.

For instance, if the application is stuck in an infinite loop or suffering from a deadlock, a liveness probe can identify this condition and trigger a container restart to recover the application's functionality.

To implement a liveness probe, you define a periodic check that sends requests to a specified endpoint within the container. If the probe receives a successful response (HTTP status code 200-399) from the endpoint, the container is considered healthy.

However, if the probe does not receive a successful response within a predefined timeframe, the container is marked as unhealthy, prompting Kubernetes to take appropriate action based on the container's restart policy.

2. Readiness Probes

Readiness probes focus on determining whether a container is ready to handle incoming requests. They are particularly useful during the startup phase of an application when it needs some time to initialize and become fully operational.

Until the readiness probe indicates that the container is ready, Kubernetes will not send traffic to that particular pod.

Like liveness probes, readiness probes involve sending requests to a specified endpoint or executing a command within the container. If the probe receives a successful response, the container is marked as ready to receive traffic.

Conversely, if the probe fails to receive a successful response within a defined time period, the container is considered not ready, and Kubernetes removes it from the list of endpoints for the associated service, effectively stopping traffic to that container.

3. Startup Probes

Startup probes are a more recent addition to Kubernetes health checks. Their primary purpose is to determine when an application has started successfully during its startup phase.

The key difference between startup probes and readiness probes is that while a readiness probe starts as soon as the container begins running, a startup probe can be delayed, allowing the application more time to initialize.

Startup probes prove beneficial for applications that may take longer to start or need to perform time-consuming initialization tasks.

By utilizing a startup probe, you can ensure that Kubernetes only considers the container as ready when the application has successfully completed its startup process.

Kubernetes Liveness Probe Settings

Understanding the settings of a liveness probe in Kubernetes is crucial for effectively monitoring the health of your containers. The liveness probe configuration includes the following settings:

1. initialDelaySeconds: The number of seconds to wait before the first liveness probe is performed after the container starts.
2. periodSeconds: The number of seconds between consecutive liveness probes.
3. timeoutSeconds: The number of seconds after which the liveness probe times out. If the probe takes longer than this value to return, it is considered a failure.
4. successThreshold: The minimum number of consecutive successful probe results required to mark the container as healthy after having previously failed.
5. failureThreshold: The number of consecutive probe failures required to mark the container as unhealthy after having previously been healthy.
6. terminationGracePeriodSeconds: Defines the amount of time the kubelet (the Kubernetes node agent) should wait between triggering the shutdown of a failed container and forcefully terminating it. This period allows the container some time to clean up and terminate gracefully.

Other Liveness Probe Settings

1. Type (httpGet, tcpSocket, or exec)

• httpGet: Performs an HTTP GET request to a specified endpoint in the container. The probe considers the container healthy if it receives a successful HTTP response code (2xx or 3xx).
• tcpSocket: Attempts to open a TCP connection to a specified port on the container. The probe considers the container healthy if the connection is successful.
• exec: Executes a specified command inside the container. The probe considers the container healthy if the command exits with a success status code (0).

2. HTTP GET Probe Settings (if applicable)

• path: The URL path for the HTTP GET request. For example, /healthz.
• port: The port on which the HTTP server is listening inside the container.
• httpHeaders: Optional custom HTTP headers to include in the GET request.

3. TCP Socket Probe Settings (if applicable)

• port: The port to which the TCP socket should attempt to connect inside the container.

4. Exec Probe Settings (if applicable)

• command: The command to be executed inside the container. It should be specified as a list of strings.

 

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
    - name: my-app-container
      image: my-app-image
      ports:
        - containerPort: 8080
      livenessProbe:
        httpGet:
          path: /healthz
          port: 8080
          httpHeaders:
            - name: Custom-Header-1
              value: Value-1
            - name: Custom-Header-2
              value: Value-2
        initialDelaySeconds: 15
        periodSeconds: 10

 

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
    - name: my-app-container
      image: my-app-image
      ports:
        - containerPort: 8080
      livenessProbe:
        exec:
          command:
            - sh
            - -c
            - curl -sS http://localhost:8080/healthz || exit 1
        initialDelaySeconds: 15
        periodSeconds: 10

 

Protect slow starting containers with startup probes

Sometimes, you have to deal with applications that require additional startup time on their first initialization. In such cases, it can be tricky to set up liveness probe parameters without compromising the fast response to deadlocks that motivated such a probe. The solution is to set up a startup probe with the same command, HTTP or TCP check, with a failureThreshold * periodSeconds long enough to cover the worst case startup time.

So, the previous example would become:

ports:
- name: liveness-port
  containerPort: 8080

livenessProbe:
  httpGet:
    path: /healthz
    port: liveness-port
  failureThreshold: 1
  periodSeconds: 10

startupProbe:
  httpGet:
    path: /healthz
    port: liveness-port
  failureThreshold: 30
  periodSeconds: 10

Thanks to the startup probe, the application will have a maximum of 5 minutes (30 * 10 = 300s) to finish its startup. Once the startup probe has succeeded once, the liveness probe takes over to provide a fast response to container deadlocks. If the startup probe never succeeds, the container is killed after 300s and subject to the pod's restartPolicy.

Define readiness probes

Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load large data or configuration files during startup, or depend on external services after startup. In such cases, you don't want to kill the application, but you don't want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services.

Note:

Readiness probes runs on the container during its whole lifecycle.

Caution:

The readiness and liveness probes do not depend on each other to succeed. If you want to wait before executing a readiness probe, you should use initialDelaySeconds or a startupProbe.

Readiness probes are configured similarly to liveness probes. The only difference is that you use the readinessProbe field instead of the livenessProbe field.

readinessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy
  initialDelaySeconds: 5
  periodSeconds: 5

Configuration for HTTP and TCP readiness probes also remains identical to liveness probes.

Readiness and liveness probes can be used in parallel for the same container. Using both can ensure that traffic does not reach a container that is not ready for it, and that containers are restarted when they fail