Observability Stack 2026 — Monitoring, Logging and Tracing for Mid-Market Companies

Anyone running software in production in 2026 faces a problem that didn't exist ten years ago: systems are distributed, ephemeral, and polyglot. A single customer request today typically passes through five to fifteen services, a handful of databases, several caches, a message queue, and at least one external API call. When that request is answered slowly or with errors, looking at a single log file is no longer enough. This is exactly where observability comes in — the discipline of instrumenting a system so that its internal states become intelligible from the outside. For mid-market companies, the topic has grown from "nice to have" to "indispensable" over the past two years: cloud migrations, Kubernetes rollouts, and availability pressure make a robust observability stack the foundation of any serious IT operations. This article shows which building blocks a modern 2026 stack contains, which open-source and commercial options have proven themselves, what costs you should realistically expect, and where the typical pitfalls lie. For context within the overall strategy, see our Cloud & DevOps Guide for Mid-Market Companies.

Monitoring vs. Observability — the distinction matters

The two terms are frequently used interchangeably in the market; that is incorrect. Monitoring describes the observation of pre-defined metrics with pre-defined thresholds — CPU load above 80 percent, disk below 10 percent free, response time above two seconds. This works excellently for known problems, but fails at what constitutes the majority in operating distributed systems: unknown problems that nobody anticipated. Observability reverses the logic. Instead of asking "is everything within defined thresholds?", it asks "can I ask this system any question and get an answer?". For this to work, you need not just more data, but structured, interconnected data.

In practice this means: an alert in an observability system is not the end, but the beginning. Whoever sees an anomaly jumps from the metrics dashboard to the matching trace, from there to the logs of the involved services, from there to the profiling information of an individual process — all linked via shared identifiers such as trace ID, service name, and deployment version. A pure monitoring system shows that something is on fire. An observability system shows what, where, why, and since when it is on fire. This difference determines whether a recovery from an outage takes minutes or hours.

For mid-market companies, the pragmatic middle ground makes sense: not every system needs full observability from day one. A static marketing website is well served by basic monitoring. As soon as microservices, Kubernetes, or a customer-critical platform are involved, investment in a complete observability stack pays off — typically within the first six to twelve months after a cloud migration or platform modernization.

The three pillars plus events

Classically, observability is built on three pillars; in recent years a fourth has been added that is enormously helpful in practice.

Pillar	Answers	Typical data	Storage behavior
Metrics	How much? How fast? How often?	Numeric time series, low cardinality, high frequency	Small per data point, many data points, long retention feasible
Logs	What happened?	Structured or unstructured text events	Large per entry, shorter retention, high I/O demand
Traces	Where did it take long or fail?	Distributed request traces with span hierarchy	Very large with full recording, hence sampling is common
Events	What changed?	Deployments, feature flags, config changes, releases	Small, high business relevance for correlation

The fourth pillar — events or change events — is worth its weight in gold in practice. Over 70 percent of all production incidents are directly related to a change that happened shortly before: a deployment, feature flag toggle, migration, or library update. Anyone who captures events in a structured way and overlays them in their dashboards sees cause and effect immediately rather than after a long search. Tools like Grafana, Datadog, and Honeycomb have dedicated overlay functions for this that many organizations leave unused.

What matters is linking the pillars through shared identifiers. In practice this means: every log entry contains the current trace ID, every trace links to the corresponding logs, every metric knows the service name and deployment label. This linkage is not a given — it only arises through consistent instrumentation with a uniform standard. This is exactly where OpenTelemetry comes in.

OpenTelemetry as the 2026 standard

OpenTelemetry — OTel for short — has been the uncontested de facto standard for instrumentation since 2023. It is a CNCF project, unifies the earlier OpenTracing and OpenCensus initiatives, and provides SDKs for all relevant programming languages as well as a powerful Collector that transforms data and forwards it to any backend. The decisive advantage: applications are instrumented once with OpenTelemetry SDKs and can then be connected to any backend — Prometheus, Grafana, Datadog, Honeycomb, New Relic. Switching the backend is possible without changing the application code.

For mid-market companies this means concretely: choosing a backend is no longer a 10-year commitment. Anyone who starts today with Grafana Cloud or a self-hosted stack and switches to Datadog in three years only needs to change the Collector configuration. This dramatically reduces the risk of tool selection and is one of the most important reasons why OpenTelemetry has spread so quickly.

In practice, a clear rollout path is worthwhile: Phase 1 instruments new services natively with OpenTelemetry SDKs. Phase 2 migrates existing services via auto-instrumentation — for Java, Python, Node.js, and Go this works with minimal code changes. Phase 3 replaces legacy agents such as StatsD, Fluentd, or direct CloudWatch calls with the OpenTelemetry Collector. Only then is the stack fully vendor-neutral and the backend interchangeable.

Metrics stack — Prometheus, VictoriaMetrics, Mimir, Thanos

In the metrics space, Prometheus has been the reference implementation for years. It works excellently up to a medium size — typically several million active time series — and then struggles with scaling, high availability, and long-term retention. It is precisely for these weaknesses that several extensions have emerged in recent years that are increasingly standard in the mid-market.

• Prometheus — the standard for pull-based metrics scraping, ideal for Kubernetes environments, simple configuration, active community
• VictoriaMetrics — drop-in-compatible alternative with significantly better storage efficiency and horizontal scaling, growing strongly in the mid-market
• Grafana Mimir — horizontal, multi-tenant long-term storage behind Prometheus, suitable for larger environments with multiple teams
• Thanos — the older, equally proven solution for global view and long-term retention across multiple Prometheus instances

Pragmatic recommendation for mid-market environments: start with native Prometheus, observe storage and CPU load over the first six months, and switch to VictoriaMetrics or Mimir once a single Prometheus instance is no longer sufficient. An early move to a distributed architecture is often over-engineering — the complexity costs more operational effort than it gains in scalability.

Logs — Loki, Elastic/OpenSearch, ClickHouse

In the logs space, the market has moved significantly over the past three years. The classic choice Elasticsearch is operationally demanding, resource-hungry, and expensive compared to modern alternatives. OpenSearch is the open fork after the AWS conflict with Elastic and for many mid-market applications the natural choice when the feature set of Elastic is needed. Both are well suited for full-text search, complex queries, and compliance use cases.

A significantly more lightweight alternative is Grafana Loki. Loki only indexes labels, not log content, and stores the actual log data in object storage such as S3 or MinIO. This makes Loki 10 to 30 times cheaper to operate than Elasticsearch — with a considerably simpler architecture. The trade-off is limited full-text search performance, which is irrelevant for most observability use cases because logs are queried in combination with traces and metrics.

For very large log volumes, ClickHouse has been the rapidly growing alternative since around 2024. Platforms such as SigNoz, Uptrace, and Quickwit use ClickHouse as a storage backend and deliver impressive query performance at low storage costs. For mid-market companies, ClickHouse is only worthwhile when Loki hits its limits — typically in the single-digit terabyte range per day.

Traces — Tempo, Jaeger, Zipkin

Distributed tracing is the third pillar and in many mid-market setups the least mature. Three open-source options dominate the market. Jaeger is the older CNCF solution, very stable, well documented, well integrated with Kubernetes. Zipkin is the even older Twitter solution, somewhat more limited in feature set, widely used in Java environments. Grafana Tempo is the newest of the three and follows a similar logic to Loki: traces are stored in object storage, the index is minimal, and correlation with metrics and logs happens via trace IDs in Grafana dashboards.

In new setups, Tempo is today the most commonly chosen option because it integrates seamlessly into a Grafana-centric stack and is significantly cheaper to operate than Jaeger. In any case, a well-thought-out sampling strategy is important — recording all traces completely is not affordable with production traffic. Tail-based sampling, where only failed or slow traces are stored in full, is the standard solution and easily enabled in OpenTelemetry Collector configurations.

Dashboards — Grafana and Kibana

In the dashboard space, Grafana has been the uncontested standard for years and is stronger than ever in 2026. It natively supports all relevant data sources, has mature alerting functions, a huge plugin ecosystem, and an active community. Anyone running an open observability stack practically cannot avoid Grafana.

Kibana remains relevant in environments that have invested heavily in the Elastic stack — there it delivers deep full-text search functions and security use cases such as SIEM analysis that are only available to a limited extent in Grafana. For pure observability, Grafana is the more robust choice; for combined observability and security analytics use cases, a hybrid with Kibana is worthwhile.

Commercial platforms — Datadog, New Relic, Honeycomb

Alongside the open-source ecosystem, there is an established field of commercial platforms that deliver all three pillars from a single source and score points through high out-of-the-box quality. Datadog is by far the broadest platform, covering Metrics, Logs, Traces, RUM, Synthetic Monitoring, and Security in a single UI, and is strongly represented in the mid-market. New Relic plays in the same segment, has had a more transparent pricing model since 2020, and is cheaper for many application profiles. Honeycomb is the specialist platform for event-based, high-cardinality observability and is very popular in engineering-driven organizations — somewhat of a niche product, but very strong in its domain.

Other relevant vendors include Dynatrace with a strong APM and infrastructure focus, Splunk with an emphasis on logs and SIEM, and Elastic Cloud as the managed variant of the Elastic stack. The selection depends heavily on the existing tool landscape, the language mix of the applications, and the size of the platform organization.

Self-hosted vs. SaaS — the cost calculation

The question "operate yourself or buy as a service" is not the question "free or expensive". Self-hosted has no license costs, but high personnel and infrastructure costs. SaaS has clear license costs, but practically no operational burden. The following table shows typical annual costs for mid-market environments.

Option	License costs/year	Infrastructure	Personnel effort
Self-hosted open-source (20–50 hosts)	€0	€15,000–30,000	20–40% of one FTE
Self-hosted open-source (50–200 hosts)	€0	€40,000–90,000	50–100% of one FTE
Grafana Cloud (managed)	€15,000–50,000	—	10–20% of one FTE
Datadog/New Relic (50–200 hosts)	€40,000–180,000	—	10–20% of one FTE

More important than the numbers is the logic behind them: SaaS platforms often charge per host and per event volume. Anyone who is careless with log volume or metric cardinality can quickly double the bill. Self-hosted penalizes the same mistake with infrastructure costs and performance problems. Discipline in instrumentation is the central lever in both variants and is closely connected to the topic of Cloud Costs and FinOps.

Alerting, SLIs, and SLOs

An observability stack without well-thought-out alerting is an expensive data collection exercise. The best practice established in recent years is based on SLIs and SLOs: a Service Level Indicator is a concrete measurement — response time, availability, error rate — and a Service Level Objective is the target value for it, for example "99.5 percent of responses under 800 milliseconds over 30 days". The SLO automatically yields an error budget, which in turn forms the basis for alerting: alarms fire when the error budget is consumed too quickly, not on every individual slow request.

• Define the SLIA concrete measurement from the user's perspective, such as the proportion of successful responses under a latency threshold. Avoid pure infrastructure metrics like CPU load as the primary indicator.
• Set the SLOA realistic target value with a time window, such as 99.5 percent over 30 days. Not every service needs to achieve 99.99 percent — higher targets cost exponentially more.
• Track the error budgetThe rate at which the error budget is consumed is the alert metric, not an individual outlier. Burn-rate alerts are standard in Prometheus, Grafana, and Datadog.
• Define escalation clearlyWho is alerted, when, and how? On-call rotation, escalation levels, and quiet hours belong in a written policy, not in Slack channels.
• Avoid alert noiseEvery alert must lead to an action. Alerts that do not lead to action get ignored — including the important ones. Regular alert reviews belong in every quarter.

The SLO methodology originally comes from the Google SRE book and is today widespread well beyond pure SRE organizations. For mid-market companies, it becomes relevant from the moment multiple teams are working on production systems and availability discussions would otherwise remain vague. For detail see our cluster on SRE in the Mid-Market.

The cardinality trap

The most common stumbling block in young observability programs is cardinality explosion. Prometheus, VictoriaMetrics, and all comparable systems store metrics as a combination of name and label set. Every unique combination is its own time series. A metric http_requests_total{service="api", endpoint="/users", status="200"} with three labels and manageable value counts yields perhaps a few hundred time series. If a label like user_id or request_id is added — one that can take millions of different values — the time series explode along with storage, index, and query performance.

Practical rules of thumb: use labels only for values with fewer than around 100 possible distinct values. Values such as user ID, session ID, trace ID, or URL with parameters belong in logs or traces, not in metrics. Before adding any new label, estimate the maximum cardinality. Tools like Grafana Mimir and VictoriaMetrics offer built-in cardinality analysis, which should become a mandatory exercise in the quarterly review.

This discipline sounds academic, but in many mid-market setups it is the difference between a performant observability stack for €2,000 per month and a sluggish system for €20,000 per month. Cardinality is the largest invisible cost driver in observability platforms.

Reepa's recommendation for mid-market stacks

From our consulting practice, the following stack has proven robust and economical for mid-market companies without a dedicated platform team: OpenTelemetry SDKs in all production services, an OpenTelemetry Collector as the central telemetry hub, and Grafana Cloud or a self-hosted Grafana LGTM stack (Loki, Grafana, Tempo, Mimir) for the first two to three years. This keeps the tool selection vendor-neutral, the operational effort manageable, and a later switch to Datadog, New Relic, or a self-hosted variant possible at any time.

For very small environments of fewer than around 20 hosts, Grafana Cloud Free Tier is often sufficient and a good starting point without investment risk. For larger environments with a clear engineering team, a self-hosted LGTM stack on Kubernetes is worthwhile — tightly integrated with the other cluster workloads, see our cluster on Kubernetes in the Mid-Market. For high-load platforms or compliance-driven industries, Datadog or Dynatrace is the pragmatic standard.

Regardless of the specific toolset: the stack lives or dies by the instrumentation discipline of the development teams. A perfect stack with poorly instrumented services delivers no usable telemetry. Invest early in training, code templates, and a clear observability standard within the organization — the tool selection is secondary.

Frequently asked questions