Live Metrics - Pods

Overview

Pod Live Metrics displays a searchable and filterable list of the Pods on the Kubernetes cluster. The table displays:

Status: The current status of the Pod
Name: The name of the Pod
Node: The Node on which the Pod is running
Namespace: The Namespace in which the Pod is running
Memory/CPU: A mini graph that displays a snapshot of the current memory/CPU usage
Memory/CPU Usage: A numerical readout of the current memory/CPU usage in gigabytes/CPU cores
Memory/CPU Request: A numerical readout of the current memory/CPU request in gigabytes/CPU cores
Memory/CPU Limit: A numerical readout of the current memory/CPU limit in gigabytes/CPU cores

Compare Pods

Live Metrics has the ability to compare Pods, which can be selected by clicking on the box on the left side of the row for each individual Pod. Once the Pods have been selected hit the Compare Pods button on the right side of the screen. Once the Compare Pods button has been selected, you will be brought to the Compare Screen.

There are two types of scales and three different modes to view the selected Pods.

The two scales are:

The Individual Scale: Displays the Pods memory/CPU relative to the Pods usage
The Common Scale: Adjust every Pod to be viewed on the same scale of the Pod with the highest memory/CPU data

The three modes are:

Grid Mode: Displays the memory/CPU on a continuously updating grid with lines for the Pods memory/CPU Request, Limit, Average, and Maximum. You can click on the three dots on the top right of the individual Pod to either get more details about the Pod or remove it from being compared
Table Mode: Displays the Pods in a linear table similar to the Live Metrics Home Page. You can click on the three dots under the Actions column of the individual Pod to either get more details about the Pod or remove it from being compared
Single Graph Mode: Displays the Pods memory/CPU on a single chart in order to view the metrics relative to one another. You can select and deselect the Request, Limit, Average, and Maximum lines from being viewed and also the specific Pods being compared on the graph

Pod Details

Pod Data

Clicking on the name of the Pod will bring you to the Pod Data details page. This page displays the following:

Name: Pod Name
Status: Status of the Pod
Namespace: Namespace Pod is deployed to
Node: Node Pod is deployed on
IP: Kubernetes IP of the Pod
UUID: Pods Universally Unique Identifier
Containers: List of Containers in the Pod
Amount of restarts: Number of times the Pod has restarted
Images: Image names for each container in the Pod
Connection Data: A detailed list of connections the pod has over the last 60 seconds.

The Connection Data table displays the following for both Sent and Recieved tabs:

Endpoint: A Pod name or Service (with Namespace) name of the Pod or Service that has communicated with the current Pod in the last 60 seconds.
Max Latency: The maximum latency (roundtrip) over the last 60 seconds between the current Pod and the Pod or Service on the list.
50x: A total count of HTTP 500 responses over the last 60 seconds.
40x: A total count of HTTP 400 responses over the last 60 seconds.
20x: A total count of HTTP 200 responses over the last 60 seconds.

By clicking on the API List next to each Pod or Service a popup window will appear. This window shows a more detailed view of the communications between the respective Pods and Services. Each API that has been communicated with in the past 60 seconds will appear, along with a detailed list of the maximum latency and counts for HTTP resonses for each API endpoint:

API: The API endpoint that has communicated with the current Pod in the last 60 seconds.
Max Latency: The maximum latency (roundtrip) over the last 60 seconds between the current Pod and the API endpoint on the list.
50x: A total count of HTTP 500 responses over the last 60 seconds.
40x: A total count of HTTP 400 responses over the last 60 seconds.
30x: A total count of HTTP 300 responses over the last 60 seconds.
20x: A total count of HTTP 200 responses over the last 60 seconds.

Real-Time Metrics

Real-Time Metrics displays a live view of metrics for Pods and their Containers. The drop down menu on the top left of the screen allows you to select which Container's information is being displayed. You can select Pod View for an aggregate view of all Containers in the Pod, an individual Container, or any combination of Containers.
Above the graph you can select the following information to view:
- Memory: Metrics around the Pods/Containers usage of RAM in gigabytes
- CPU: Metrics around the Pods/Containers usage of CPU/vCPU cores in core counts
- Threads: Metrics around the Pods/Containers current total open threads count
- Throttling: Throttling is a precentage of times tasks in a cgroup have exhausted all of their available time as specified by the CFS (Completly Fair Schedular) quota before their quota is reallocated/reset
- Voluntary Context Switches: Metrics around the Pods/Containers voluntary context switches. Voluntary context switches occur when a task blocks because it requires resources that are unavailable
- Non-Voluntary Context Switches: Metrics around the Pods/Containers non-voluntary context switches. Non-voluntary context switches take place when a task executes for the duration of its time slice and then is forced to relinquish the processor

On the bottom on the graph you can adjust the slider to adjust the length of time for the displayed information. Slide it to the left to view more data from time before what is currently displayed and to the right to display more information after the information that is currently displayed. You can also drag the slide to the left of right to adjust the information from the times being displayed.

Below the slider display you can select or de-select the following from the Pods/Containers to either display or hide the information:

Request: The resource request for the Pod/Containers
Limit: The resource limit for the Pod/Containers
Average: The average usage for the Pod/Containers for the past 24 hours, or length of the Pods existence if less than 24 hours
Maximum: The maximum usage for the Pod/Containers for the past 24 hours, or length of the Pods existence if less than 24 hours

Network Connections

Network Connections displays the Pods network connections for the following:

Listen: Represents waiting for a connection request from any remote TCP and port
SYN-Sent: Represents waiting for a matching connection request after having sent a connection request
SYN-Received: Represents waiting for a confirming connection request acknowledgment after having both received and sent a connection request
Established: Represents an open connection, data received can be delivered to the user. The normal state for the data transfer phase of the connection
FIN-Wait-1: Represents waiting for a connection termination request from the remote TCP, or an acknowledgment of the connection termination request previously sent
FIN-Wait-2: Represents waiting for a connection termination request from the remote TCP
Close-Wait: Represents waiting for a connection termination request from the local user
Closing: Represents waiting for a connection termination request acknowledgment from the remote TCP
Last-ACK: Represents waiting for an acknowledgment of the connection termination request previously sent to the remote TCP (which includes an acknowledgment of its connection termination request)
Time-Wait: Represents waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request
Closed: Represents no connection state at all

You can hover over each connection status to see a description. You can filter by Status or Amount. The Pie Chart represents the ratio between different types of connections. Click on a sector or on a label below the chart to see a description of each connection type.