agent-swarm.devagent-swarm.dev
Guides

Investigating a stale partner data feed with AI agents

Agent Swarm compares a partner feed's source timestamps with the expected update interval and hands engineering and operations a source-linked review packet.

Agent Swarm can be configured so a shared-agent workflow runs the freshness checks a team defines against one partner data feed, and assembles a source-linked review packet for engineering and operations. Deterministic steps compare the feed's source timestamps with the expected update interval and record the last successful ingestion; an agent puts each result beside the record behind it and the time it was read. People verify whether the feed is stale enough to act on, and decide what happens next. The workflow removes the assembly: collecting one feed's evidence from a pipeline and a partner's status page, then getting the same picture in front of two teams.

The scope is one running feed and whether it is still delivering what it did: the timestamps on what arrived, the last successful ingestion on the team's side, and the handoff when the two disagree. It does not cover a planned change to a partner API, an individual booking or order discrepancy, listings that disagree with the approved catalog (that case), sales pipeline records before a forecast, or a purchase-order confirmation.

When another tool fits better

Monitoring you already run on the pipeline. If your ingestion job reports lag and last-run status, and an alert reaches an on-call engineer when a feed goes quiet, keep it. It sees the pipeline directly, and an agent workflow over the same signals adds a review step to a page someone already watches.

The partner's own delivery report. Some feeds ship with a status page stating when the last file was sent. When the question is only whether the partner sent anything, that report answers it.

Direct human triage. One feed, one owner, an obvious failure: if a single person watches the feed and can see it stopped, a message to the partner is the whole job.

Agent Swarm fits a different shape. The expected interval sits in a partner agreement, the ingestion log in one system, and the internal records that consume the feed in another. Operations uses the data and engineering runs the pipeline, so the judgement needs evidence from both. Choosing an AI operating model sets out where that kind of work belongs.

An illustrative workflow

This is an illustrative adaptation, not a built-in feed-monitoring feature and not a customer result. It needs custom integrations, thresholds the team agrees, and permissions the team grants and tests.

The example is a fictional property marketplace availability feed, where a partner pushes updated availability per unit on an agreed interval.

1. The team agrees what fresh means for this feed. One feed, the update interval the partner commits to for availability, and a threshold for how far past that interval still counts as late. The interval and the threshold sit beside the check, so a later packet is read against the rule it ran under. A schedule sets the cadence inside a workflow that carries the steps below.

2. A configured script compares the timestamps and records the last successful ingestion. It reads the newest timestamp the feed carries and the last ingestion on the team's side, compares each with the expected interval, and emits JSON. The same inputs produce the same result on every run, and no model performs the comparison. Scripts reach each source through script connections, with credentials from the credential broker, so no raw secret enters script source.

3. The check sets both sides beside each other and flags what a bare threshold hides. The source timestamp sits next to the internal record timestamp, each with its read time. Then the cases one comparison misses: a null or absent timestamp on either side, a record with no internal counterpart, and a timestamp with no timezone or an offset that changes the verdict. A three-hour gap reads as stale in UTC and current in the feed's own zone.

4. An agent assembles the packet. One line per finding, with a link to the source, the value read and when it was read. This is the split the Reports from multiple sources playbook documents: deterministic collection in code, then an agent writing from that JSON. It is also the part that needs a custom source skill for this partner, written by the team.

5. A person verifies at a gate. A human-in-the-loop gate pauses the workflow, posts a summary linking the full packet, and resumes only on a human response. Three outcomes follow: pause use of the data, contact the partner, or no action with the result recorded for the next run.

Worked example

The rows below are illustrative. No company or partner is named; none comes from a customer case.

SignalWhat the check comparesEvidence attachedQuestion, and to whom
The last partner update is older than the agreed intervalThe newest source timestamp against the agreed thresholdBoth timestamps, the interval applied, when each was read"Is this feed late, or has the partner changed its schedule?" to the engineering owner. "Can we keep serving on the data we hold?" to the operations owner
An availability record the feed carries is missing internallyThe unit identifiers in the source availability extract against the internal table for the same windowThe unmatched unit identifiers on each side, the extract's timestamp"Did the load drop these rows, or does the partner send them elsewhere?" to the engineering owner. "Are these the records the team works from?" to the operations owner
A timestamp has no timezone, or an offset that changes the answerThe raw timestamp and its zone against the same interval in that zoneThe raw string, the zone assumed, both readings, when read"Which zone does the partner publish in?" to the engineering owner, who asks the partner contact. "Which reading did we plan against?" to the operations owner
The source timestamp is unknownWhether a newest-record timestamp exists at allThe empty field, with the endpoint and the read time"Can the timestamp come from delivery metadata?" to the engineering owner. "Who at the partner can confirm the last send?" to the operations owner

One packet, two readers. The engineering owner takes the rows about the load and the connection, the pipeline facts. The operations owner takes the rows asking whether the data is safe to work from today. The open questions stay visible in the packet.

Unavailable evidence and escalation

A check is only as complete as what it can reach. Four gaps recur: the feed endpoint does not answer, the ingestion log sits behind a missing permission, the partner status page requires a login, and a record the comparison needs was never written.

Each gap appears in the packet by name, with what was attempted and who can grant access or answer. That line's freshness verdict is marked unknown: silence from a source never becomes a "fresh" conclusion, because a check that could not read a timestamp has learned nothing about freshness.

Then escalation. The gate carries a timeout and a fallback, per the HITL gates guidance: a nudge before the deadline, a named backup owner, and the item held openly with its reason. Nobody fills a gap by pasting a timestamp into a chat.

Limits

A recent timestamp is not proof of accurate data. It says something arrived. It does not say the rows are complete, the values are right or the fields the team relies on are populated. A feed can deliver on time and still be wrong, and this check will not catch that. A periodic check narrows the window between a feed going stale and someone noticing; it does not close it, since a feed can go stale immediately after a run.

No feed monitoring ships with agent-swarm. There is no built-in partner-feed watcher, no connector for a specific partner and no default threshold. Every piece here is configuration the team builds.

No repair. The workflow does not re-trigger ingestion, does not backfill and does not pause anything on its own. It reports, and a person acts in the system that owns the action.

No promised outcome. There is no customer case behind this pattern and no measured saving in time or headcount. Whether it earns its setup depends on what the team relies on the feed for.

What stays human-owned

  • Verifying freshness. Judging that a gap is real, that an explanation holds and that the data is stale enough to matter is a person's call with a name on it.
  • Pausing use of the data. Stopping work that depends on the feed, or telling downstream teams to hold, is an operational decision with consequences.
  • Contacting the partner. Every question about a schedule or an unannounced change goes through the person who owns that relationship.
  • Communications and corrections. What the team tells its own users, and whether anything is recalculated, belongs to people.

The agent reports what it read and what it compared. A named owner decides. Gate the irreversible step, let the reversible ones run: reading a feed is reversible, telling a customer the data is current is not.

Building blocks

Each piece ships in agent-swarm and is documented. Pointing it at one partner feed is the illustrative part.

  • Reports from multiple sources: deterministic swarm-script and script-run collection that emits JSON, then an agent writing the narrative from it.
  • HITL gates: the blocking human-in-the-loop node, its branches, and the timeout and fallback.
  • Script connections and scripts credential broker: how a script reaches a partner system or a warehouse while the secret stays in the broker.
  • Workflows: the nodes that carry the check, the packet and the gate.
  • Scheduling: the cron or interval behind the check's cadence.

On this page