Can You Trace One Failed Action Across Your Self-Hosted App Stack? A Log-Correlation Checklist
Use a request or correlation ID, consistent timestamps and a controlled test to see whether logs from your proxy, application, workers and dependencies tell one useful story.
Map the action before searching the logs
Start with one user-visible failure and sketch the route its work may take. A common investigation map includes the reverse proxy, application, background worker, database and any external dependency involved. Treat this as a hypothesis: a particular application may use only some of these components, or may handle work differently.
For each step, note which service produces logs, where those logs can be accessed, and what event would indicate that the action arrived or completed. This gives you a bounded search instead of a broad scan through unrelated messages.
- Write down the action, approximate time and expected outcome.
- List the components that might handle it, including any asynchronous worker.
- Record how to access each component’s logs and whether the logs can be searched by time or field.
- Mark gaps explicitly: a component with no accessible logs cannot contribute to the trail.
Choose an identifier and check where it survives
A request ID or correlation ID can provide a common value to search for across related events. Do not assume every service accepts, forwards or records one. Check the actual configuration and logs for each component in your path.
If an incoming request already has an identifier, decide whether to preserve it or create an identifier at a boundary you control. Avoid treating a client-supplied value as trusted evidence without checking how your application handles it. The practical test is whether the same chosen value appears in the relevant records, not whether a setting merely exists.
- Choose a field name and format your components can consistently record.
- Check the proxy and application first, then each downstream component.
- For components you cannot configure, note what alternative fields can help narrow the search, such as a time window or job identifier.
- Do not include credentials, tokens or personal information in the identifier.
Carry the trail into queued work
A user action may finish its web request before a queued task starts or completes. In that case, the request log alone may not show the outcome. Check whether your application passes a correlation value into the queued job and whether the worker records it when starting, finishing or failing the task.
If the job system uses its own job identifier, record the relationship between that identifier and the original request or correlation ID. Verify this behavior with the application and queue configuration you actually run; support and field names vary.
- Confirm whether the enqueue event records a job ID and correlation ID.
- Check whether the worker logs a start and a final outcome for the same job.
- If the correlation ID is not carried through, document the available link between the request and job.
- Distinguish a job that was never enqueued from one that was enqueued but did not complete.
Make timestamps and log fields comparable
A useful trail needs records you can compare. Check what timezone each service uses, whether timestamps include a timezone, and whether clocks appear to disagree. If events seem out of order, treat that as a question to investigate rather than assuming the displayed order reflects the actual sequence.
Consistent, structured fields make filtering easier. Datadog’s documentation notes that logs should be JSON-formatted or parsed by the appropriate language-level log processor before correlating logs with traces. That guidance concerns trace correlation; for a log-only investigation, verify that your own search tools can reliably identify the fields you need.
- Look for a timestamp, service or component name, severity, event message and correlation field.
- Check whether timestamps identify their timezone and whether the services’ clocks appear aligned.
- Prefer a stable field name for the correlation value across components.
- Test how your log viewer parses the fields; do not assume a value in unstructured text is searchable as a field.
Run a controlled failure test
Before relying on the trail during an incident, test it with a low-risk action in a controlled environment or at a time when you can safely observe the result. Use a known identifier, trigger a failure you can recognize, and then search each component’s logs. Do not create a failure that could affect important customer data or business operations.
The test is successful only if you can reconstruct a useful sequence: where the action entered, which component handled it, whether work was queued, and where the failure became visible. If the trail stops, record the missing handoff and decide whether you can add logging or need another source of evidence.
- Use a test action with an expected, observable failure outcome.
- Search by the known correlation ID and compare the records across components.
- Check that the worker’s result can be linked back to the initiating action.
- Write down gaps, ambiguous timestamps and fields that could not be searched.
Keep the trail useful without over-collecting data
Diagnostic logs help investigate behavior and failures, but they can also contain sensitive information if applications or configurations write it there. Review the fields your test exposes, and avoid logging credentials, access tokens, or personal data that is not needed for the investigation.
Decide who needs access to logs and how long they need to remain available for your operational needs. These choices depend on your application, risk and governance requirements; do not assume that hosting or infrastructure management automatically settles them. Airbip manages cloud infrastructure around self-hosted applications and automates routing and TLS certificates, but customers still need to understand their own log access and retention arrangements.
- Inspect actual log examples before broadening collection or sharing access.
- Limit access to people who need logs for operational or security work.
- Set retention deliberately and review whether it remains suitable for your needs.
- Recheck logging after application or configuration changes.
Know what logs can—and cannot—establish
A correlated diagnostic trail can help explain how a failure unfolded, but it is not automatically a complete record of who performed an action or whether a record is authoritative. If you need reliable evidence of actions for accountability or governance, identify and maintain the appropriate audit records separately.
Document the limits of your trail: components that do not expose logs, fields that are not propagated, gaps in time coverage, and events that cannot be linked with confidence. A clear account of what the logs establish is more useful than treating a correlation ID as proof of every step.
- Use diagnostic logs to investigate operation and failure.
- Identify whether separate audit records are required for your accountability needs.
- Record known gaps and avoid claiming a complete timeline when the evidence is incomplete.
- Repeat the controlled test when important components or logging arrangements change.
Frequently asked questions
Do all services need distributed tracing to correlate a failed action?
No. This checklist focuses on whether related log events can be linked with an identifier and compared across components. It does not assume every service supports distributed tracing. Check what each component actually records and document gaps.
What if a queued job does not include the original request ID?
Check whether the enqueue event records a job ID and whether the worker logs that same job ID. If there is no reliable link back to the initiating request, note the gap and assess whether the application or job configuration can record one.
Should application logs be JSON?
Structured logs can make fields easier to identify and search. Datadog’s documentation specifically says logs should be JSON-formatted or parsed by the appropriate language-level log processor when correlating logs with traces. Verify what your own log tools can parse and search.
Are diagnostic logs the same as audit records?
Do not assume they are interchangeable. Diagnostic logs are used to investigate system behavior and failures. If you need an authoritative record of actions for accountability or governance, determine what separate audit records are appropriate for your application and requirements.
Does managed infrastructure take care of log access and retention?
Do not assume so. Check the actual arrangement for your application and hosting setup, including who can access logs and how long they are retained. Managed infrastructure does not remove the need to understand your data and governance responsibilities.
Sources and further reading
- Correlate Logs and Traces — Datadog
- Application logs and agent traces in one view — Braintrust