The examination


The whole examination, and what each part leaves out.

Expertluma examines one version of a system, for one job the institution names. The institution brings the cases and the expected answers. Those answers are not sent to the system under examination. A named person at the institution decides. What cannot be observed is recorded as not examined.

The job and the version

The institution names the system, the version, and the job. A result belongs to that pair. If either changes, the earlier result does not cover the new one. A new version name, on its own, does not show that the system changed.

The cases

The institution brings the cases and the expected answers. They are not sent to the system under examination. Confidential cases are not accepted until access to those answers inside the product has been verified. The cases are not production records. This examination does not take production records and does not de-identify them. If production records are all you have, the examination waits.

The call

The system's owner gives a test endpoint and a credential they can revoke. Each case is sent and the reply is stored. A connection failure is recorded as a connection failure, never as the system being wrong. If the endpoint is down, or the expected answers are not in, the examination is not finished.

The comparison

Where a reply is compared with the institution's answer, the comparison uses a fixed rule. Today that covers a class, a field, or a citation. The record says, for each compared case, whether the reply matched, and which facts the vendor only declared. Whether the cases were kept out of training is a declaration. The examination does not prove it, and the record says so.

What the record does not cover

If the job depends on a tool call, a retrieval control, or an enforcement control inside the institution's system, and the examination did not see it, the record names it as not examined. The examination does not watch the system afterwards, and it does not sit in the path that deploys it.

The decision

The named person reads the record and decides: use it, send it back, or not yet. Expertluma does not approve, fix, or sell the system. The fee does not change with the decision.

The next version

The institution changes the system in its own environment. A new version is a new examination. The earlier record stays with the earlier version. See re-examination.

Plain answers

Does one miss fail the examination? The record shows every compared case against the institution's answer. The institution decides what a miss means for the job.

Agents, voice, and specialist domains? Examination agents are designed to work inside a mandate. They do not decide, and that mode is not running. Today a scored comparison covers a class, a field, or a citation. Ten specialist domains are a catalogue, not ten finished examinations.

Do you see production data? No. Cases the institution is allowed to give, through a test endpoint. Not production records, and not patient information.