Semantic Conventions for Feature Flags in Logs

Status: Experimental

This document defines semantic conventions for recording feature flag evaluations as a log record emitted through the Logger API. This is useful when a flag is evaluated outside of a transaction context such as when the application loads or on a timer.

Motivation

Feature flags are commonly used in modern applications to decouple feature releases from deployments. Many feature flagging tools support the ability to update flag configurations in near real-time from a remote feature flag management service. They also commonly allow rulesets to be defined that return values based on contextual information. For example, a feature could be enabled only for a specific subset of users based on context (e.g. users email domain, membership tier, country).

Since feature flags are dynamic and affect runtime behavior, it’s important to collect relevant feature flag telemetry signals. This can be used to determine the impact a feature has on a request, enabling enhanced observability use cases, such as A/B testing or progressive feature releases.

Recording an Evaluation

Feature flag evaluations SHOULD be recorded as attributes on the LogRecord passed to the Logger emit operations. Evaluations MAY be recorded on “logs” or “events” depending on the context.

Evaluation event

The table below indicates which attributes should be added to the LogRecord and their types.

Status:

The event name MUST be feature_flag.evaluation.

Defines feature flag evaluation as an event.

A feature_flag.evaluation event SHOULD be emitted whenever a feature flag value is evaluated, which may happen many times over the course of an application lifecycle. For example, a website A/B testing different animations may evaluate a flag each time a button is clicked. A feature_flag.evaluation event is emitted on each evaluation even if the result is the same.

Attribute	Type	Description	Examples	Requirement Level	Stability
feature_flag.key	string	The lookup key of the feature flag.	logo-color	Required
error.type	string	Describes a class of error the operation ended with. [1]	provider_not_ready; targeting_key_missing; provider_fatal; general	Conditionally Required [2]
feature_flag.variant	string	A semantic identifier for an evaluated flag value. [3]	red; true; on	Conditionally Required [4]
feature_flag.context.id	string	The unique identifier for the flag evaluation context. For example, the targeting key.	5157782b-2203-4c80-a857-dbbd5e7761db	Recommended
feature_flag.evaluation.error.message	string	A message explaining the nature of an error occurring during flag evaluation.	Flag header-colorexpected typestringbut found typenumber``	Recommended [5]
feature_flag.evaluation.reason	string	The reason code which shows how a feature flag value was determined.	static; targeting_match; error; default	Recommended
feature_flag.provider_name	string	Identifies the feature flag provider.	Flag Manager	Recommended
feature_flag.set.id	string	The identifier of the flag set to which the feature flag belongs.	proj-1; ab98sgs; service1/dev	Recommended
feature_flag.version	string	The version of the ruleset used during the evaluation. This may be any stable value which uniquely identifies the ruleset.	1; 01ABCDEF	Recommended

[1] error.type: If one of these values applies, then it MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
flag_not_found	The flag could not be found.
invalid_context	The evaluation context does not meet provider requirements.
parse_error	An error was encountered parsing data, such as a flag configuration.
provider_fatal	The provider has entered an irrecoverable error state.
provider_not_ready	The value was resolved before the provider was initialized.
targeting_key_missing	The provider requires a targeting key and one was not provided in the evaluation context.
type_mismatch	The type of the flag value does not match the expected type.
general	The error was for a reason not enumerated above.

[2] error.type: If and only if an error occurred during flag evaluation.

[3] feature_flag.variant: A semantic identifier, commonly referred to as a variant, provides a means for referring to a value without including the value itself. This can provide additional context for understanding the meaning behind a value. For example, the variant red maybe be used for the value #c05543.

[4] feature_flag.variant: If feature flag provider supplies a variant or equivalent concept.

[5] feature_flag.evaluation.error.message: If and only if an error occurred. It’s NOT RECOMMENDED to duplicate the value of error.type in feature_flag.evaluation.error.message.

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
_OTHER	A fallback error value to be used when the instrumentation doesn’t define a custom value.

feature_flag.evaluation.reason has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
cached	The resolved value was retrieved from cache.
default	The resolved value fell back to a pre-configured value (no dynamic evaluation occurred or dynamic evaluation yielded no result).
disabled	The resolved value was the result of the flag being disabled in the management system.
error	The resolved value was the result of an error.
split	The resolved value was the result of pseudorandom assignment.
stale	The resolved value is non-authoritative or possibly out of date
static	The resolved value is static (no dynamic evaluation).
targeting_match	The resolved value was the result of a dynamic evaluation, such as a rule or specific user-targeting.
unknown	The reason for the resolved value could not be determined.

Body fields:

Body Field	Type	Description	Examples	Requirement Level	Stability
value	undefined	The evaluated value of the feature flag.	#ff0000; 1; true	Conditionally Required [1]

[1] value: If and only if feature flag provider does not supply variant or equivalent concept. Otherwise, value should be treated as opt-in.