Assessment and AI policy

• The standard is public. Anyone can read what a role is measured against before they are measured.
• Every score shows its evidence. A score cites the parts of the conversation it rests on, so it can be checked and challenged.
• Only the job is assessed. Never personality, voice, accent, appearance, identity or health.
• People decide. A verdict is evidence for a hiring decision, never the decision itself.

Every role has a gold standard: what someone in that role needs to know and do on day one. Each standard is built from the published certification frameworks for the role, set by a physician, and names its sources and the date it was last reviewed.

A standard is made of competencies, grouped into five dimensions: clinical skills, workflow and tools, patient communication, compliance and safety, and role knowledge. A standard also lists critical failures: things a person in the role must never do, such as an action that would put a patient at risk.

You can read every standard on our role standards pages. When a standard changes, the date on it changes too, and a scorecard always records the standard it was measured against.

Before each assessment, we ask you to confirm three things:

• that you agree to the conversation being transcribed, so it can be scored and so you can see what was heard;
• that you understand you are speaking with AI, and that AI scores what you say;
• that you are alone and will answer from your own experience, keeping real patients' and employers' details out of it.

You can practice first. Practice sessions work the same way, are scored so you can learn from them, and are never shown to employers.

The examiner

A readiness assessment is a spoken conversation of up to 18 minutes. An AI examiner, running on an OpenAI model, presents situations drawn from the standard for your role and follows up on what you say. It leads the conversation; it does not teach, hint or grade you while you talk. It never asks for real patient information, and steers you away from it if you start to give it.

Your answers

Answer as yourself, without help from another person, notes or another tool. Describe what you would do, not who you did it for. Your voice is streamed to the model so it can hear and transcribe you, and is not stored. The transcript is what is kept and scored.

Your transcript

After the assessment you can read the transcript, and correct a small number of words where it misheard you. Corrections are limited so the transcript stays a faithful record of what was said.

Levels first, then numbers

Each answer is first rated on four levels for that competency: Not yet, Developing, Proficient or Exemplary. Only then is it given a number, consistent with that level. Every score cites the turns of the conversation it rests on.

Competency scores make up a score for each of the five dimensions, and the dimensions make up your overall score, out of 100. Each is always shown beside the bar it was measured against, which is 70 unless the standard sets another.

A critical failure counts only if it names a failure the standard lists and cites something you actually said. Anything else the scorer reports is dropped rather than trusted.

Keeping scoring steady

The scorer is shown answers that people have already graded for the same competencies, and it is tested against those graded answers. A change to the model or its instructions that would move verdicts is caught before it reaches anyone.

Three verdicts

• Ready Day One. You met the bar for this role on the evidence you gave.
• Partially Ready. You scored at least 50, and targeted practice on your weakest areas closes the gap.
• Not Ready. This attempt did not yet show the evidence the role needs, or an answer crossed a line the standard treats as unsafe.

When a pass is held

Clearing the bar is needed for Ready Day One, but it is not enough on its own. A score above the bar is held at Partially Ready when:

• the assessment did not cover enough of the standard;
• one dimension fell too far below the bar to pass on an average;
• the evidence was not strong enough to certify.

A critical failure makes the verdict Not Ready whatever the score. Your scorecard always says which of these applied, and which turn of the conversation it rests on.

The examiner asks you to confirm you are answering alone, follows up in ways a script cannot predict, and notes, without accusing, signs that an answer is being read or that another voice is present. The scorer judges these signs from the words of the transcript alone, never from how you sound.

If an attempt shows signs it was answered with help, your score is kept but the credential it would earn is held until you sit the assessment again. A held attempt is never treated as proof of cheating on its own. If we find an attempt broke our terms, we may withdraw its results and any credential it earned, and we will tell you why.

The models come from OpenAI. They receive only what each task needs, and OpenAI does not use what we send to train its models.

Only what you can do for the role is assessed. The examiner and the scorer are instructed never to assess personality, emotion, accent, voice, culture, identity, health or any protected trait, such as race, religion, sex, age or disability.

The examiner never ends an assessment because of nerves, silence, an accent, a language difficulty or a disability. We do not use facial analysis, and we do not analyze your voice for anything but the words.

A verdict is evidence, not a hiring decision. Employers decide who to interview and who to hire, and they are responsible for following employment law when they do, including any rules where they hire on tools that help make hiring decisions. MedTality does not hire, reject or guarantee a placement for anyone.

If "Apply for me" is on, the verdict decides only which roles we apply to on your behalf. It never decides whether an employer hires you.

If you do not pass, your scorecard names the gaps, and a learning pathway works on them.

• Attempts. Each assessment allows up to 3 attempts, unless the app shows another number for it.
• Waiting time. At least 24 hours between attempts, so there is time to practice.
• What counts. An attempt counts once the examiner finishes it. Leaving early costs you nothing.
• Once you pass. The assessment is complete, and you keep the verdict you earned.

Passing an assessment earns a credential for that role, with a code anyone can check. A Gold Standard Credential does not expire. A credential from an organization's program lasts as long as the program sets, and you are reminded before it does. A credential is not a license to practice.

The readiness assessment is spoken. If a disability or another reason makes a spoken assessment hard for you, tell us through Support before you start, and we will work out an adjustment with you, such as more time or another way to answer. We use what you tell us only to make the adjustment. Our accessibility page explains what works today.

If you think a verdict is wrong, reach us through Support within 30 days. Say which assessment and which part you think is wrong. A person will review the transcript and the evidence each score cites, and tell you what they found.

If the review finds a mistake, we correct the scorecard and the verdict, and restore any attempt the mistake cost you.

Laws on AI and automated tools in hiring are growing, such as New York City's rules on automated employment decision tools, Colorado's law on high-risk AI systems, and Illinois's rules on AI in employment decisions. We design MedTality to support them: the standards are public, every score shows its evidence, candidates are told AI is involved before they start, and people make every hiring decision.

Employers who use MedTality where these laws apply remain responsible for their own duties under them, and we provide the information about how scoring works that they need.

For questions about how assessments work or how we use AI, ask us through Support.