Skip to main content
Gå til innhold

LogQL best practices

The practices on this page help you write LogQL queries that run faster and use fewer resources in our Loki setup.

Understanding the basics​

Structured metadata vs stream labels​

Before you read the practices, make sure you understand these two concepts:

  • Stream labels are the labels in curly braces, for example {service_name="nginx"}.
  • Structured metadata is additional data attached to logs, and you filter on it with pipe operators such as | key="value".

Essential best practices​

1. Start small, then expand​

Always test your queries on a small time range first, so you don't overuse resources by accident. Start with 5 minutes or 1 hour, and expand to larger ranges once the query works.

2. Use labels to filter first​

Loki indexes labels but doesn't index log content. Start every query with a label selector. A query that selects one service's streams by label is fast:

{service_name="myapp"} |= "error"

A query that selects every stream is very slow, because it searches all logs in the time range:

{service_name=~".+"} |= "error"

Loki refuses a query whose selector has no label matcher, such as {}, or has only matchers that also match an empty value, such as {service_name=~".*"}.

3. Order matters for performance​

Place filters before parsing operations. Here the filter runs first, and the query is fast:

{service_name="nginx"}
| detected_level="error"    # Filter first
| json                       # Parse second

With the two steps swapped, the same query is slow:

{service_name="nginx"}
| json                       # Parse first (slow!)
| detected_level="error"    # Filter second

4. Use simple string matching when possible​

The exact-match operator |= is faster than the regex operator |~. Use a regex only when you really need pattern matching. An exact match is fast:

{service_name="nginx"} |= "404"

A regex is slower:

{service_name="nginx"} |~ "40[0-9]"

5. Parse only what you need​

JSON and logfmt parsing are expensive, so only parse when necessary. This query is good, because it uses a string filter to keep only the relevant lines and then parses a single field:

{service_name="api"}
|= "orderId"           # Filter to relevant logs
| json orderId         # Parse only the field you need
| orderId="12345"

Avoid parsing everything, as this query does:

{service_name="api"}
| json                 # Parses all fields in every log!
| orderId="12345"

Working with metrics and aggregations​

6. Aggregate after filtering​

When you use functions such as count_over_time or rate, filter your data first. Counting only the error lines is efficient:

count_over_time(
  {service_name="nginx"}
  |= "error"                  # Filter first
  [5m]                        # Then count per 5 minutes
)

Counting without a filter is inefficient:

count_over_time({service_name="nginx"}[5m])  # Counts everything, even non-errors

7. Use small time windows​

For aggregation functions, use the smallest window that gives you the resolution you need. A 5-minute window is good:

sum(rate({service_name="payment"} |= "error" [5m]))

A 1-hour window is resource-intensive:

sum(rate({service_name="payment"} |= "error" [1h]))

Advanced tips​

8. Avoid high-cardinality labels​

Don't use labels with millions of unique values (like trace_id, user_id, session_id) as stream labels. Keep them in structured metadata instead.

9. Structure complex queries in stages​

For complex queries, think in three stages:

  1. Narrow down with stream labels and simple filters.
  2. Parse only the fields you need from the filtered logs.
  3. Aggregate the results to summarize them.

For example:

count_over_time(
  {service_name="payment"}          # 1. Narrow with labels
  |= "timeout"                      # 1. Further narrow with string filter
  | json status                     # 2. Parse only status field
  | status="500"                    # 2. Filter parsed data
  [5m]                              # 3. Aggregate
)

10. Watch your stream count​

Each unique combination of labels creates a stream. If your query matches too many streams (50,000+), it will be slow regardless of other optimizations.

This query counts the streams that logged in the last minute. Widen [1m] to match your query's time range:

count(count_over_time({service_name="myapp"}[1m]))

Quick reference: do's and don'ts​

DoDon't
Start with label selectorsSelect every stream with {service_name=~".+"}
Filter before parsingParse before filtering
Use |= for exact matchesUse |~ regex unnecessarily
Test on small time rangesStart with 30-day queries
Aggregate after filteringAggregate raw data
Keep high-cardinality data in structured metadataUse high-cardinality stream labels

Additional performance considerations​

11. Avoid querying during peak hours​

If you're running deep scans, for example across days or multiple services, try off-peak hours to avoid throttling and to reduce the impact on other users.

12. Query cost awareness​

Make sure developers understand that log queries can be expensive, and encourage them to:

  • use metrics instead of logs for high-frequency monitoring on dashboards
  • reserve logs for debugging or audit trails
  • use Explore for ad hoc queries and dashboards for focused queries
Dashboard auto-refresh

Dashboard queries auto-refresh frequently, so avoid putting expensive queries in dashboards.

Need more help?​

  • For getting started with Grafana, see Using Grafana
  • Contact the Platon team (#platon-observability) if you need assistance optimizing specific queries