ETSI CIM · Testing Task Force

Interoperability tests with real brokers

SEAMWARE · 2 November 2026, rewritten 6 November 2026 on top of IOP_TP · a companion to All of NGSI-LD in the test suite by September 2027

The suite already has Test Cases with several real brokers: the interoperability Test Cases in IOP_TP, written for the 2026 test event. They cover Create Entity, Retrieve Entity and Query Entities over four broker topologies. This page builds on them: what they do, what a run shows, what they need next, and a long list of what else several real brokers could test, each item as a new IOP Test Case.

1. What the suite has today

Distributed operations against a mock

One broker under test, at url in resources/variables.py. Every other broker is a stub served by the HttpCtrl mock server on port 8086, told apart by a path prefix (/broker1, /broker2).
develop has 91 such Test Cases in TP/NGSI-LD/DistributedOperations: 21 for queries and retrieval, 67 for provision and 3 for loop detection (a Via header that already names the broker).
They test how one broker forwards a request, against a partner the test controls exactly. They stay as they are.

Interoperability: `IOP_TP`

A folder of its own next to TP: IOP_TP/NGSI-LD/Interoperability, 24 robot files, each run twice, with the core @context and with the test suite's @context: 48 Test Cases.
Up to five real brokers, b1_url to b5_url, passed on the command line. The brokers are started outside the suite and start empty. The client always talks to b1.
Two keywords in resources/IOPUtils/InteroperabilityUtils.resource: Compose IOP Configuration takes a list of (entity id, registration file, mode, source broker, broker it is created on) and creates the registrations; Delete Registrations And Entities removes them and deletes the test's entities on every broker with local=true.
Data: two OffStreetParking and two Vehicle entities, full and in parts, in data/entities/interoperability; 13 registrations in data/csourceRegistrations/interoperability (inclusive, exclusive, redirect, auxiliary, some limited to attributes or to one entity id).
Tags: iop, the topology (cnf_01 to cnf_04), the registration modes (additive-inclusive, additive-auxiliary, proxy-exclusive, proxy-redirect) and the context.
Names: IOP_<operation>_<topology>_<variant>_<context>. Operation 001 = Create Entity, 002 = Retrieve Entity, 003 = Query Entities (GET and POST).
The answers are checked against what the leaf brokers return with local=true, so no assertion depends on one implementation's data.
A viewer that draws a Test Case's topology is on the branch iop-tests-viewer (tools/interoperability_visualizer), not yet in develop.

The four topologies

Topology	Brokers	Registrations	Test Cases
`cnf_01`	3	b1 → b2 inclusive, b1 → b3 exclusive	001_01_01-02, 002_01_01-02, 003_01_01-02
`cnf_02`	4	b1 → b2 inclusive, b1 → b3 redirect, b1 → b4 redirect	001_02_01-02, 002_02_01-02, 003_02_01-02
`cnf_03`	4	b1 → b2 auxiliary, b1 → b3 inclusive, b1 → b4 inclusive	001_03_01-02, 002_03_01-02, 003_03_01-02
`cnf_04`	5	b1 → b2 auxiliary, b1 → b3 inclusive; b2 → b4 redirect, b2 → b5 redirect; b3 → b5 exclusive. Two hops.	001_04_01-02, 002_04_01-02, 003_04_01-02

Within a topology the registration files differ from one Test Case to the next (all attributes, a few attributes, one entity id), so the 24 files are 24 different setups.

2. What a run shows

Run on 6 November 2026 against five copies of one implementation, develop as of that day: 42 of 48 Test Cases pass, 21 of the 24 files. The three failing files:

IOP_003_02_02: the Test Case checks that b2's own OffStreetParking:1 equals b3's, and that b3's own OffStreetParking:2 equals b4's, but its setup creates different versions there (the full entities in b2 and b3, location and name only, with other locations, in b3 and b4). Each broker's own copy is to be compared with the data created on it.

A Test Case issue, twice. b1's answer is right: the location with observedAt wins, as TS 104-175 clause 8 says.

IOP_002_04_01: b1 is expected to return OffStreetParking:1 with the location held in b5. The only routes to b5 are b2's redirect for location, which b1 does not reach because its auxiliary registration towards b2 is limited to availableSpotsNumber and totalSpotsNumber, and b3's exclusive registration, which is exclusive-1: OffStreetParking:2, totalSpotsNumber.

Looks like a Test Case issue: 002_04_02 uses exclusive-3 (OffStreetParking:1, location) in the same place and passes.

IOP_003_03_02: a query by type=OffStreetParking,Vehicle on b1 also returned Vehicle:2, held in b4, whose registration covers OffStreetParking only.

That part of the Test Case is right and the fix belongs in the implementation: a forward never asks a source for more than its registration covers. With it fixed, a second check fails: b1's Vehicle:1, from a query with attrs=location, is compared with the full entity instead of the location-only one the Test Case builds two lines before. A Test Case issue.

One documentation slip as well: IOP_001_03_01's description says the entity "shall not be present in b3"; the Test Case checks, rightly, that it is not in b2.

3. What IOP_TP needs next

Much of what an earlier version of this page asked for is already there: several broker URLs, a keyword that creates registrations between brokers, entities created on a chosen broker, a teardown over every broker, brokers started outside the suite, a tag. What is missing:

Starting the brokers. Each implementation's own way today. A script, or a docker compose file per implementation, that brings up N copies on N ports and prints the --variable list; a mixed run is the same with a different list. This is task T4 of the TTF, Interoperability Test Automation via Scripted Orchestration.
The Test Cases that fail today (§ 2) fixed, so that a full run is a clean baseline.
Registration files with more in them: operations other than redirectionOps, management.timeout, tenant, contextSourceInfo, jsonldContext, location, expiresAt. Compose IOP Configuration takes them as they are.
More entities: the RV (§ 4), entities for paging (60 or so), a LanguageProperty, linked entities for join. The OffStreetParking and Vehicle files stay.
Subscriptions: the notification server on 8085, reached from b1 only, is enough for subordinate subscriptions; the Test Case reads the subordinate subscriptions on b2 to b5 directly.
A recording proxy between two brokers, for the few Test Cases that check a forwarded request (Via, the Entity Map location passed on, contextSourceInfo headers).
Stopping a broker in the middle of a Test Case (one broker down during a write): a hook in the start script.
Five brokers stay the limit. Every item below fits in b1 to b5.
Operation numbers beyond 003, one per operation, and topologies from cnf_05 on, each described once (in the README or the viewer) and reused.

4. The first new configuration: the RV

The RV is the case this page started with. In IOP_TP terms it is two new configurations, cnf_05 and cnf_06, with the client talking to b1 as in every IOP Test Case.

Split entity, two types

A recreational vehicle is both a vehicle and a building. Its vehicle part lives in one broker and its building part in another. A third broker, the one the client talks to, has a registration for each of them, both for the RV with both of its types (an EntityInfo type can be an array, TS 104-175 § 5.2.6.6.2).

Broker b2:  { "id": "urn:ngsi-ld:RV:1", "type": "Vehicle",  "speed":  { "type": "Property", "value": 80 } }
Broker b3:  { "id": "urn:ngsi-ld:RV:1", "type": "Building", "floors": { "type": "Property", "value": 1 } }

Broker b1:  inclusive registration urn:ngsi-ld:RV:1 / ["Vehicle", "Building"] -> b2
            inclusive registration urn:ngsi-ld:RV:1 / ["Vehicle", "Building"] -> b3

NGSI-LD has no way to ask for only part of an entity. Whatever the request, when the RV is in the answer it is the entire RV: "type": ["Vehicle", "Building"], with speed and floors.

Retrieve without a type, GET /entities/urn:ngsi-ld:RV:1 → the entire RV.
Retrieve with one type, GET /entities/urn:ngsi-ld:RV:1?type=Building → the entire RV.
Query without a type but with the entity id, GET /entities?id=urn:ngsi-ld:RV:1 → the entire RV.
Query with one type, GET /entities?type=Building → the entire RV. It matches through its building part, and the vehicle part comes with it.
Query with both types, GET /entities?type=Vehicle,Building → the entire RV, once, not twice.
The same queries with an Entity Map (entityMap=true), on the first page and on the next ones → the entire RV. The map holds every registration for urn:ngsi-ld:RV:1, b2's as well as b3's.
A type neither part has, ?type=Car → 404 for the retrieve, no RV in the query.

Filters over both parts

GET /entities?type=Building&q=speed>50 → whether the RV matches depends on whether the filter is applied before or after the parts are put together (splitEntities, TS 104-175 § 4.3.3.3.3). Both settings get a Test Case.

Three brokers, no stubs. The Test Case checks only what b1 answers, so b1, b2 and b3 can each be any implementation. Entities and registrations are new data files next to the OffStreetParking ones; the keywords are the existing ones.

The same RV, registrations of different width

The case the Entity Map exists for. b1's registration towards b2 is narrow, Vehicle only. The one towards b3 is wide, both types:

Broker b1:  inclusive registration urn:ngsi-ld:RV:1 / "Vehicle"               -> b2
            inclusive registration urn:ngsi-ld:RV:1 / ["Vehicle", "Building"] -> b3

GET /entities?type=Vehicle&entityMap=true&splitEntities=true on b1:

b2 has the RV as a Vehicle, so urn:ngsi-ld:RV:1 goes into the Entity Map.
b3's registration also matches type=Vehicle, so it is in the map entry for urn:ngsi-ld:RV:1 too, although b3 holds the RV only as a Building.
Each part is fetched by the entity's id, without the type: b2 gives the vehicle part, b3 the building part. Asked for type=Vehicle, b3 would have nothing, since it holds the RV only as a Building.
The answer is the entire RV: "type": ["Vehicle", "Building"], speed and floors.

The same holds on every following page, and for the map read back through GET /entityMaps/{id}: the entry for urn:ngsi-ld:RV:1 lists both registrations.

The rule behind it: the type chooses the registrations and the entities; the parts of a chosen entity are then fetched by its id. It follows from TS 104-175 § 9.6: the query is "a unitary operation, masking the fact" that the data is distributed, so b1 answers as one broker holding the whole RV would; and with split entities the map holds "candidate Entities", checked on the assembled entity, not on each part.

Open for discussion

In the same setup, GET /entities?type=Building: b2's registration (Vehicle only) does not match, so b2 is not asked and the answer is only the building part of the RV, "type": "Building" and floors.

That is fine: the registrations decide which sources a query reaches, and a deployment that wants the entire RV for type=Building registers b2 for both types. The answer still holds everything the matching sources have.
It is not: a broker holding the whole RV would return it entire, so the distributed answer differs from the central one. To avoid that, b1 would look up every registration for the RV's id once the RV has matched, whatever their types.

Either way, a client cannot ask for only the part of type X: NGSI-LD has no such query. The Test Case for type=Building is written once the group has chosen.

5. What we could test

A long list, grouped. The number in brackets is how many brokers a Test Case needs; b1 is always the one the client talks to. IOP_TP marks what the existing Test Cases already cover, with the file. Every item is run with both contexts, as today.

Split entities: one entity, parts in several brokers

Retrieve and query an entity whose attributes are spread over b1's sources by inclusive, exclusive, redirect and auxiliary registrations (3–5) IOP_TP 002_0x, 003_0x
attrs naming only a remote attribute: only that broker's part in the answer (3–5) IOP_TP 002_02_02, 002_03_02, 002_04_02, 003_02_02, 003_03_02
The RV by every route: retrieve with and without a type, query by id, by one type and by both types, with and without an Entity Map: always the entire RV (3) — cnf_05
The RV with a narrow registration (Vehicle, towards b2) and a wide one (both types, towards b3): ?type=Vehicle with an Entity Map gives the entire RV, on every page, and the map lists both registrations (3) — cnf_06
The same with the wide registration written as idPattern plus type, instead of the RV's id (3)
The RV asked for by a type neither part has: 404 for the retrieve, an empty query result (3)
The RV with a third part in b1 itself: local and remote parts merged (3)
The same attribute in two brokers with different datasetIds: both instances in the answer (3)
The same attribute and datasetId in two brokers: the merge rule of § 8.5 decides which one is returned (3)
An auxiliary source with an attribute b1 already holds: b1's value wins; an attribute only the auxiliary source has is added (2) — partly in IOP_TP 002_03_01
A remote part whose expiresAt is in the past: discarded, the rest returned (3)
An entity split over four brokers, one attribute each, merged by b1 (5)
pick and omit over a split entity: applied after merging (3)
A split entity in concise, keyValues and GeoJSON: the same merge in every format (3)
lang over a LanguageProperty held remotely (2)

Queries across brokers

Query by type, GET and POST, over entities in several brokers (3–5) IOP_TP 003_01 to 003_04
Query by two types (3–4) IOP_TP 003_02_02, 003_03_02
q over two attributes held in two brokers, splitEntities=true: matched after assembly (3)
The same with splitEntities=false: the filter is forwarded and the entity does not match (3)
georel with the GeoProperty in one broker and the queried attribute in another (3)
scopeQ over a scope held remotely (2)
A list of ids whose entities live in three different brokers (4)
idPattern matching entities in several brokers (4)
count=true: each entity counted once, however many brokers hold a part of it (3)
Paging with an Entity Map over 60 entities in four brokers: every entity exactly once, across pages (5)
Paging while a remote entity stops matching between pages: the page is shorter, not padded (3)
An expired Entity Map: recreated, paging continues (3)
local=true: no broker is asked, only b1's own data (3)
csf: only the matching registrations are used (4)
A registration with a location: a geo-query outside it does not reach that broker (3)
Registrations on type only, on id only, on idPattern only and on attribute names only (2 each)
A registration that does not list queryEntity among its operations: not asked (2)

Several hops and federations

Two hops, b1 → b2 → b4/b5 (5) IOP_TP cnf_04
A chain of five, b1 → b2 → b3 → b4 → b5: the answer from the far end, unchanged (5)
A tree: b1 over four domain brokers, a query by type that each domain has entities of (5)
A geographic federation: four domains registered by location, a geo-query that only two of them overlap (5)
The Entity Map's linked maps over two hops: page 2 uses the map b2 created on page 1 (3)
A split entity whose parts are one and two hops away (4)
join/joinLevel with the linked entity in another broker (3)
containedBy with linked entities over three brokers, and a cycle among them (4)

Loops, for real

b1 → b2 → b1 with inclusive registrations: the query ends, the entity comes back once (2)
A triangle b1 → b2 → b3 → b1: the same (3)
A loop of redirect registrations: 508 Loop Detected (TS 104-176 § 6.3.18) (2)
Via: after three hops the header names all three brokers, each name checked against that broker's /info/sourceIdentity (3, proxy)
A loop on Create Entity: the entity is created once (2)
A loop on a subscription: subordinate subscriptions do not multiply (2)

Writes across brokers

Create Entity with inclusive, exclusive, redirect and auxiliary registrations, one and two hops: each attribute ends up where the registrations say (3–5) IOP_TP 001_01 to 001_04
Create Entity with a redirect registration for the whole type: all in b2, nothing in b1 (2)
Append Attributes to a split entity: each attribute in the broker registered for it (3)
Update and Partial Attribute Update of an attribute held remotely (2)
Replace Entity and Replace Attribute over a split entity (3)
Merge Entity over a split entity, including urn:ngsi-ld:null deletions (3)
Delete Attribute: removed where it is held, the others untouched (3)
Delete Entity: removed from every broker that holds a part (4)
One broker down during a write: 207, with the parts that worked and the one that did not (3)
Batch create, upsert, update, merge and delete over entities that land in four brokers, 207 where some fail (5)
Purge Entities over several brokers (4)
Set Attribute Value on a remote attribute (2)

Subscriptions across brokers

A subscription on b1 for a type held in b2: b2 gets a subordinate subscription, a change in b2 reaches the subscriber (2)
A change to the part of a split entity in b3: the notification carries the whole entity (3)
A registration added after the subscription: the new broker gets a subordinate subscription too (3)
A registration deleted: its subordinate subscription goes away (2)
The subscription deleted, paused or updated on b1: the same in every subordinate subscription (3)
A subscription with q over attributes in two brokers (3)
A subscription with a geo-query over a federation of four domains (5)
Notification over MQTT from a remote change (2, plus an MQTT broker)
Registration subscriptions on b1, notified when b2 registers itself (2)

Temporal

Query Temporal Evolution with one attribute's history in b2 and another's in b3 (3)
Retrieve Temporal Evolution of a split entity (3)
timerel and lastN applied after merging (3)
Temporal Entity Maps: paging over histories in several brokers (4)
Aggregated temporal representation over a split entity (3)

Discovery and registration details

/types and /types?details=true: every broker's types, each once (3)
/attributes, with and without details, over several brokers (3)
/types/Vehicle with Vehicle entities in two brokers (3)
contextSourceInfo: the headers it names reach b2 on every forwarded request (2, proxy)
The registration's tenant: b1's tenant a reaches b2 as tenant b (2)
The registration's jsonldContext: used on the forwarded request, the answer compacted back with the client's context (2)
management.timeout: a slow b2 is given up on, b3's data still returned (3)
An expired registration: no longer used (2)
/info/sourceIdentity on every broker of the topology: all different (5)

Mixed implementations

This is what IOP_TP was written for, and it needs no new Test Cases: the same Test Cases with each of b1 to b5 a different implementation. With the start script of § 3 it becomes a matrix instead of one result: for each Test Case, which implementation sat where. The ones worth running first:

The RV with b1, b2 and b3 three different implementations, in every order (3)
An Entity Map created by one implementation and used by another one hop on (3)
Subordinate subscriptions created by one implementation in another (2)
The Via identities of three implementations in one chain (3)

6. How it could go in

The same rules as the main plan: small MRs, at most about 15 Test Cases each, each one passed against at least one topology of real brokers before it is sent.
MR 1: the three failing files of § 2 and the description of IOP_001_03_01, so that a full run is a clean baseline. Ready: with it, 48 of 48 pass against five copies of an implementation that respects the registrations.
MR 2: the start script with a compose example, and the iop-tests-viewer branch merged if its authors agree.
MR 3: the RV, cnf_05 and cnf_06, as the first new Test Cases, so the group can judge the direction on one example.
Then one MR per group of § 5, in the order the group finds most useful.

7. What Seamware asks of the group

The authors of IOP_TP: a look at the Test Case issues of § 2.
Agreement that new multi-broker Test Cases go into IOP_TP, with its names, tags and keywords.
A decision on the discussion in § 4: the building part only, or the entire RV, for type=Building with a narrow registration towards b2.
Implementations willing to take a position in the mixed runs.