AI / LAI Based Test Generation
We taught the appliance's own intelligence to read the reference material a network exposes — object identifiers, real published backbone maps, device semantics — and from it manufacture the exact known-answer test stimulus needed to prove automatic discovery and SmartHost tracking are right, then we graded the product against it.
The value and the need for test stimulus
A localized advanced intelligence placed on a network does real work: it discovers the devices, infers how the network is shaped, tracks every host, and designs what gets monitored. Because operators act on what it shows, a wrong answer delivered with confidence is worse than no answer. Discovery and inference cannot ship on the strength of "it passed its own unit tests."
That is the problem test stimulus solves. Test stimulus is the controlled, known-answer input you feed a system so its output can be graded against truth. For a network product it is the network itself: the devices, their adjacency, their forwarding tables, their responses on the wire. The need is specific:
- Self-test against ground truth, not the author's expectations. A synthetic unit test only proves the code does what its author imagined. It cannot show how inference behaves on the messy variety of real networks.
- You cannot keep hundreds of real backbones in the lab, and a customer's live network is not a place to run experiments. Stimulus is how you bring that reality into continuous testing.
- A regression gate. Once stimulus is scored automatically, every future change to discovery or tracking must keep scoring correctly or the build fails. The stimulus becomes the product's conscience.
The hard part is that good stimulus is expensive to author by hand. A single realistic switch means hundreds of internally consistent objects — interface names, address tables, forwarding entries, neighbor records, routes — all cross-referencing each other. That is exactly the kind of exhaustive, non-clever work augmentation is built for.
Using AI to read the references and create the stimulus
The method has one governing idea: the stimulus generator reads the same references the product reads, so the test and the thing under test share one source of truth and cannot silently drift apart.
Augmented engineering, not automation: the human supplies the intent and the substance, and the AI is an enablement tool that amplifies it.
This is the same pattern MCG applies across its work. In our storytelling practice, the author of Betrayal (Bil Herd, the Uncounted series) supplies the story and the AI acts purely as a story-enablement tool. Here the engineer supplies the references and the known answers, and the AI manufactures the stimulus that exercises them. The judgment stays human; the leverage is the model.
Structural reference — real published topologies
There is a large academic dataset of real service-provider and research backbones, built from maps the operators themselves published. The harness reads each graph, converts it into the neighbor-adjacency evidence a live sweep would produce, independently classifies the graph's true structure, feeds that evidence to the product's own inference, and grades the result against the known answer — across the whole corpus in one run.
Live-wire reference — the device itself
Published maps carry no forwarding tables and no address resolution, so the lower-layer half of discovery needs a device answering on the wire. The harness reads the product's own probe to learn the exact object identifiers it asks for, synthesizes a coherent device from a plain description, and serves it as genuine network-management traffic over the wire — built from the standard library alone, with no third-party agent. One definition fixes the object identifiers shared by the emulator and the product, so stimulus and product stay in lock-step by construction. No frontier model, no cloud, no data leaving the boundary: the generation is a mechanical synthesis from references the appliance already understands.
SmartHost tracking — the process we validate
SmartHost is the process that gives every host a stable identity and a physical location, independent of its IP address. It answers "what is this, and where is it plugged in?" by fusing two correlations:
- Location — joining the switch forwarding table (hardware address to switch port to segment) with address resolution (IP to hardware address) on the shared hardware address, SmartHost pins every host to a physical switch, port, and segment: the authoritative "where is this host" that a ping or port sweep cannot answer.
- Identity — a probabilistic match over durable signals (serial number, device engine identifier, host key, certificate, hardware address, system name, naming records), each weighted by how stable it is. A host that changes its IP but keeps its hardware address and switch port is a move, not a new host.
The full binding SmartHost maintains is hardware-address to IP to port to switch to segment. The live-wire stimulus was built specifically to stand that chain up end to end, and on its first run it caught two real product faults that the product's own unit tests could not see — both exposed the moment the product met a device it did not author.
What we found out — stated plainly
We are upfront about the numbers. Run against a corpus of real backbones, the first version of the inference was confidently wrong: it reported a single-hub or dual-hub for roughly seven in ten networks that have no hub at all — they are meshes — and it never once abstained. Its reported confidence was a fixed constant, so it carried no information. The stimulus made that impossible to miss, and drove the fix.
How we treat a mesh. The shape is now decided by the count of genuine forwarding cores. One core is a single hub; two cores are a dual hub; three or more co-equal interconnected cores are a full mesh — a first-class shape, no longer forced into a hub label it does not deserve. A graph with no dominant hub and no core set is honestly returned as "unknown": the inference abstains rather than fabricate a hub. Confidence is now computed from the strength of the evidence, and the full rationale is recorded so an operator can see why and overrule it.
The figures on this page are a dated snapshot of a reproducible run on a public reference dataset, not a fixed constant — they are re-measured every time the harness runs. Specific published figures are confirmed before release.
The case that matters in the field
The public corpus is service-provider and research backbones, which are overwhelmingly meshes — the hardest possible setting for hub detection, and we report it on purpose. It is not what a typical deployment runs. A real customer network — an enterprise, a municipality, a small carrier — is almost always a single-hub or dual-hub design: one or two cores with spokes homing to them.
We expect single-hub and dual-hub to be on the order of eighty percent of the deployments met in the field (an engineering expectation from the customer base, stated as such, not a lab measurement). And the hub family is exactly where the inference scores highest: a clean star classifies at ninety-six percent, a true mesh at ninety-three percent, and the inference never fabricates a hub on a graph that has none. The operator confirms the shape in every case, so the inference's job is to propose the right answer most of the time and never to assert a wrong one with confidence. It does.
The same discipline is described on our augmented engineering page: the AI extends reach, the engineer verifies the result, and nothing — here, no test verdict — is trusted without a path back to the evidence that justifies it.
Want the full method and the numbers?
The white paper lays out the technique, the transparent results, and how the validation reproduces end to end.
Download the white paper