Can You Reconcile a Failed Integration? An Evaluation Checklist for Self-Hosted Applications

Why a successful connection does not prove that two systems agree

An integration is not proven by a green connection indicator, a successful HTTP response or the absence of a visible error. Those signals may show that a request reached an endpoint, but they do not prove that the intended record was created once, updated completely, linked to the correct counterpart or reflected in both systems at the expected time.

The practical test is stronger: after a delay, failure or retry, can your team determine what happened to a specific business record and bring the systems back into agreement without guesswork? If the answer is no, the integration may work on a normal day but remain operationally unsafe when communications fail or people make changes in parallel.

RFC 9110 makes an important distinction here. Idempotency concerns whether the intended effect on the server is the same when a request is performed once or multiple times; it is not established merely by the response a client observed. Treat connection success as transport evidence, not reconciliation evidence.

• Connection evidence answers: did a request appear to complete?
• Reconciliation evidence answers: which record changed, what was the final state, and do both sides now agree?
• Operational readiness answers: who investigates exceptions, how are they corrected, and what evidence is retained?

The common failure modes to design for

Most reconciliation problems fall into a small number of patterns. Naming them before selection or launch helps teams ask better questions of an application, connector or custom workflow.

Missed records occur when an event is never collected, a delivery fails, a filter excludes it, or a later process cannot find it. Duplicates occur when a sender retries after uncertainty and the receiver treats the attempt as a new create. Partial updates occur when only some fields or dependent records are applied. Stale data occurs when delivery is delayed or a change is not captured. Conflicting edits occur when two systems or users update the same business information independently.

These patterns can overlap. A timeout may leave the sender uncertain whether a create was applied. Retrying may produce a duplicate; avoiding the retry may leave a missing record. That is why the application needs observable record identity and the team needs a documented decision path.

• Missing: a source record has no corresponding target record.
• Duplicate: multiple target records represent one source record or one event.
• Partial: a record exists but required fields, relations or downstream effects are absent.
• Stale: the record exists but does not reflect the agreed change window.
• Conflict: separate edits compete and one silently overwrites or obscures the other.

Start with authority, not technology

Before evaluating APIs, webhooks or automation tools, define the business question for every important field and event: which system is authoritative? A CRM might be authoritative for a sales owner, an ERP for an invoice status, and a form application for the original consent submission. There is no universally correct source of truth; there must be an explicit decision that matches the business process.

Write authority at the field and event level, not only at the application level. “The CRM is the master” is too vague when a marketing system is allowed to maintain subscription status or an internal tool owns an operational approval. Also define whether information travels one way, is copied for reference only, or can be edited on both sides.

Where two-sided editing is unavoidable, define a conflict rule before launch. It may be a controlled review queue, an approved precedence rule, or conditional updates that reject changes made against an outdated record version. Timestamps can help order events, but timestamps alone may not reliably detect conflicting edits. RFC 9110 describes entity tags and If-Match as a mechanism that can prevent accidental overwrites when supported by an application or API.

• For each field: authoritative system, permitted writers, target systems and sync direction.
• For each event: source event, expected target effect, acceptable delay and evidence required.
• For each conflict: detection method, decision maker and correction action.
• For deletions and merges: retention rule, propagation rule and recovery procedure.

The application evaluation checklist

Use this integration reconciliation checklist during product selection, connector design and pre-launch testing. Request a demonstration or documentation for each item using a realistic record, not a generic assurance that an integration exists.

First, require stable record identifiers. A useful identifier is non-null, unique and stable enough to match the same business record across extracts and investigations. Names, email addresses and display labels can change or be shared. Database guidance on primary keys reflects the underlying need: a primary key uniquely identifies a row and is non-null. If a product relies on a “unique” business field, ask specifically how null values are treated; uniqueness rules can still allow multiple nulls depending on the underlying implementation.

Next, assess time and history. Can you retrieve creation and update times, preferably with a clear timezone convention? Can you see who or what changed the business record, the prior and new values where needed, and the related sync attempt? An application audit trail answers a business-change question. Infrastructure and request logs answer a delivery and execution question. They are complementary, not interchangeable.

Finally, test retrieval. A reconciliation process needs repeatable exports or documented API/import methods that can retrieve the relevant population with identifiers, statuses and change times. A flat-file export is not automatically sufficient: CSV has ambiguity around null versus empty values, and formatting choices can change comparisons. Define normalization rules and validate the extract before relying on it.

• Identifiers: stable internal ID, external or source ID storage, uniqueness behavior and null handling.
• Timestamps: created time, changed time, timezone convention and whether timestamps are consistently maintained.
• Change history: actor, action, before/after values where required and link to the affected record.
• Sync visibility: status, attempt time, target record reference and actionable error detail.
• Logs: searchable request or service evidence, correlation value and time-bounded retrieval.
• Exports and APIs: documented scoped extraction, pagination or filters, field definitions and import behavior.
• Access controls: who may view logs, exports and correction tools, and whether sensitive values need redaction.

Assess retry and duplicate handling without assuming it is safe

Do not infer retry safety from an HTTP method, a retry toggle or a vendor claim that retries are automatic. RFC 9110 advises against automatically retrying a non-idempotent request after a communication failure unless the client knows the request semantics are idempotent or can determine that the original request was never applied.

Ask for the exact duplicate-control mechanism. Does the receiving side accept an idempotency key? Can the workflow store and reuse a source event ID? Does an update target a stable record ID rather than searching by a mutable field? Is a create transformed into an upsert under defined conditions? What response, stored status or lookup lets an operator establish whether the first attempt took effect?

Test uncertainty deliberately in a non-production environment where safe. Send or simulate a delayed request, then inspect the final records, attempt history and logs. The desired outcome is not necessarily that every request is retried. The desired outcome is that the team can distinguish “not applied,” “applied once,” “applied more than once,” and “requires review.”

• Document the idempotency or deduplication key and where it is persisted.
• Confirm the receiver's behavior when the same key or source event is submitted again.
• Verify the retry limit, delay policy and terminal-failure behavior where those settings are available.
• Check whether retries can be searched by record ID or correlation value.
• Define the manual decision rule when the original request's outcome is unknown.

Design a reconciliation report that finds exceptions

A reconciliation report should be a repeatable control, not an emergency spreadsheet created after an incident. It should compare a defined population over a defined time window, using the agreed authoritative system and stable identifiers. Run it after the expected sync delay, not immediately after an event, unless the process requires near-immediate verification.

Start with totals, but do not stop there. Counts can reveal population-level differences, such as 200 source records and 197 target records. Exception lists make those differences actionable by showing the stable ID, source and target references, relevant timestamps, sync state and reason for review. Field-level comparisons then identify records that exist on both sides but differ in important values.

Use samples as a quality check alongside automated comparisons. A count can match while wrong records have been linked, while a field comparison can miss a business rule that was not represented in the extract. Select a documented sample method appropriate to the volume and risk, and retain the results with the run evidence.

• Scope: business object, time window, inclusion and exclusion rules, and expected delivery delay.
• Totals: source count, target count, matched count, missing count, duplicate count and unresolved count.
• Exceptions: stable ID, source ID, target ID, event or correlation value, timestamps, owner and disposition.
• Field checks: only authoritative or business-critical fields, with normalization rules for nulls, dates, casing and formats.
• Samples: documented selection method, reviewer, date and result.
• Sign-off: report owner, completion time and link to retained evidence.

Document the operating model before launch

Software capabilities do not substitute for ownership. Even when an application exposes identifiers, logs, exports and APIs, an application owner must decide what to monitor, who can access the evidence, how often reviews happen and how corrections are authorized.

Create a short runbook that someone other than the workflow builder can follow. NIST describes log management as an ongoing operational process, which is the right model for integrations too. A useful runbook turns a vague instruction such as “check errors” into a bounded activity with named owners, triggers and expected evidence.

For self-hosted workloads, keep the layers clear. Application history may show a business change; service logs may show process output; request-level access logs can provide evidence of handled requests. OpenTelemetry notes that trace and span identifiers in logs support correlation across distributed components. If your stack has an equivalent correlation value, carry it through the workflow and make it searchable. Where relevant, Traefik access-log configuration can provide request evidence, while Docker Compose logs can be retrieved for bounded time windows as supporting investigation evidence. These are tools to operate deliberately, not proof that the application itself has a complete audit trail.

• Name an integration owner, a business data owner and an escalation contact.
• Set a review frequency based on business impact and acceptable delay.
• Define who may retry, edit, merge, delete or re-import records.
• Specify the correction path: fix source, fix target, replay, suppress, or open a manual review.
• Set evidence retention, access and redaction requirements for reports, logs and exports.
• Define closure criteria for an exception and when a recurring defect becomes a change request.

Run a tabletop test before relying on the integration

A tabletop test is a low-risk way to prove that the reconciliation process works when the happy path does not. Use a test record or a carefully controlled non-production scenario where safe. Agree in advance what failure or delay is being simulated, who observes it and how the record will be cleaned up.

Trace one known record from the originating business action through the sending workflow, receiving application and reconciliation report. Record the source ID, target ID if created, event or correlation value, expected fields and expected time window. Then use the available application history, request evidence, service logs and scoped exports to answer whether it was applied and whether any correction is needed.

The test is successful only if the team can find the exception, make an authorized correction, verify the final state and retain enough evidence for a later reviewer to understand the decision. If the record cannot be traced, the remedy may be to add an identifier or correlation value, improve exports, change the workflow design or reduce the coupling between systems.

• Choose a controlled record and record its stable source identifier before testing.
• Introduce or simulate a delayed or failed delivery only where it is safe and authorized.
• Check whether retry behavior is observable and whether duplicate prevention works as intended.
• Run the reconciliation report after the expected delay window.
• Correct the resulting exception through the documented path, then verify both systems.
• Capture gaps in identifiers, logging, exports, permissions or ownership and resolve them before launch.