Semantic conventions for container metrics

Status: Development

Container metrics

This document describes instruments and attributes for common container level metrics in OpenTelemetry. These metrics are collected from technology-specific, well-defined APIs (e.g. Kubelet’s API or container runtimes).

Metric: container.uptime

This metric is recommended.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.uptime	Gauge	s	The time the container has been running. [1]

[1]: Instrumentations SHOULD use a gauge with type double and measure uptime in seconds as a floating point number with the highest precision available. The actual accuracy would depend on the instrumentation and operating system.

Metric: container.cpu.time

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.cpu.time	Counter	s	Total CPU time consumed. [1]

[1]: Total CPU time consumed by the specific container on all available CPU cores

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
cpu.mode		Conditionally Required [1]	string	The CPU mode for this data point. A container’s CPU metric SHOULD be characterized either by data points with no mode labels, or only data points with mode labels. [2]	user; system

[1] cpu.mode: Required if mode is available, i.e. metrics coming from the Docker Stats API.

[2] cpu.mode: Following states SHOULD be used: user, system, kernel

cpu.mode has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
idle	Idle
interrupt	Interrupt
iowait	IO Wait
kernel	Kernel
nice	Nice
steal	Steal
system	System
user	User

Metric: container.cpu.usage

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.cpu.usage	Gauge	{cpu}	Container’s CPU usage, measured in cpus. Range from 0 to the number of allocatable CPUs. [1]

[1]: CPU usage of the specific container on all available CPU cores, averaged over the sample window

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
cpu.mode		Conditionally Required [1]	string	The CPU mode for this data point. A container’s CPU metric SHOULD be characterized either by data points with no mode labels, or only data points with mode labels. [2]	user; system

[1] cpu.mode: Required if mode is available, i.e. metrics coming from the Docker Stats API.

[2] cpu.mode: Following states SHOULD be used: user, system, kernel

cpu.mode has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
idle	Idle
interrupt	Interrupt
iowait	IO Wait
kernel	Kernel
nice	Nice
steal	Steal
system	System
user	User

Metric: container.memory.usage

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.memory.usage	Counter	By	Memory usage of the container. [1]

[1]: Memory usage of the container.

Metric: container.memory.available

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.memory.available	UpDownCounter	By	Container memory available. [1]		container

[1]: Available memory for use. This is defined as the memory limit - workingSetBytes. If memory limit is undefined, the available bytes is omitted. In general, this metric can be derived from cadvisor and by subtracting the container_memory_working_set_bytes metric from the container_spec_memory_limit_bytes metric. In K8s, this metric is derived from the MemoryStats.AvailableBytes field of the PodStats.Memory of the Kubelet’s stats API.

Metric: container.memory.rss

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.memory.rss	UpDownCounter	By	Container memory RSS. [1]		container

[1]: In general, this metric can be derived from cadvisor and specifically the container_memory_rss metric. In K8s, this metric is derived from the MemoryStats.RSSBytes field of the PodStats.Memory of the Kubelet’s stats API.

Metric: container.memory.working_set

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.memory.working_set	UpDownCounter	By	Container memory working set. [1]		container

[1]: In general, this metric can be derived from cadvisor and specifically the container_memory_working_set_bytes metric. In K8s, this metric is derived from the MemoryStats.WorkingSetBytes field of the PodStats.Memory of the Kubelet’s stats API.

Metric: container.memory.paging.faults

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.memory.paging.faults	Counter	{fault}	Container memory paging faults. [1]		container

[1]: In general, this metric can be derived from cadvisor and specifically the container_memory_failures_total{failure_type=pgfault, scope=container} and container_memory_failures_total{failure_type=pgmajfault, scope=container}metric. In K8s, this metric is derived from the MemoryStats.PageFaults and MemoryStats.MajorPageFaults field of the PodStats.Memory of the Kubelet’s stats API.

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
system.paging.fault.type		Recommended	string	The paging fault type	minor

system.paging.fault.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
major	major
minor	minor

Metric: container.disk.io

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.disk.io	Counter	By	Disk bytes for the container. [1]

[1]: The total number of bytes read/written successfully (aggregated from all disks).

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
disk.io.direction		Recommended	string	The disk IO operation direction.	read
system.device		Recommended	string	The device identifier	(identifier)

disk.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
read	read
write	write

Metric: container.network.io

This metric is opt-in.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.network.io	Counter	By	Network bytes for the container. [1]

[1]: The number of bytes sent/received on all network interfaces by the container.

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
network.interface.name		Recommended	string	The network interface name.	lo; eth0
network.io.direction		Recommended	string	The network IO operation direction.	transmit

network.io.direction has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
receive	receive
transmit	transmit

Metric: container.filesystem.available

This metric is recommended.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.filesystem.available	UpDownCounter	By	Container filesystem available bytes. [1]		container

[1]: In K8s, this metric is derived from the FsStats.AvailableBytes field of the ContainerStats.Rootfs of the Kubelet’s stats API.

Metric: container.filesystem.capacity

This metric is recommended.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.filesystem.capacity	UpDownCounter	By	Container filesystem capacity. [1]		container

[1]: In K8s, this metric is derived from the FsStats.CapacityBytes field of the ContainerStats.Rootfs of the Kubelet’s stats API.

Metric: container.filesystem.usage

This metric is recommended.

Name	Instrument Type	Unit (UCUM)	Description	Stability	Entity Associations
container.filesystem.usage	UpDownCounter	By	Container filesystem usage. [1]		container

[1]: This may not equal capacity - available.

In K8s, this metric is derived from the FsStats.UsedBytes field of the ContainerStats.Rootfs of the Kubelet’s stats API.