Clinical documentation consumes 30 to 40 percent of a physician's day. AI medical scribes built on Azure OpenAI change that equation. We have deployed this approach in several healthcare settings, and the pattern is consistent: faster notes, fewer errors and physicians who finish on time.
This article explains what an AI scribe does, the architecture we use, and the parts that take the most time in practice: prompts, EHR integration and physician trust.
What an AI scribe actually does
An AI medical scribe listens to the patient and physician conversation in real time, extracts the clinical intent and drafts structured notes: SOAP notes, history of present illness, assessment and plan. The physician reviews the draft and it is inserted into the EHR. This is comprehension, not transcription. A transcript records what was said; a scribe decides what belongs in the chart and where.
The Azure OpenAI architecture
We use Azure OpenAI for note generation and Azure AI Speech for real-time audio. The pipeline runs in this order:
- Audio streams from the clinic device to Azure AI Speech
- The transcript is produced in chunks as the visit progresses
- A specialty-aware prompt turns the transcript into a structured clinical note
- The note is reviewed and edited by the physician
- The approved note is written to the EHR through its API
The services involved are HIPAA-eligible Azure services, and the design keeps data inside the customer's tenant boundary. Eligibility is a starting point, not a guarantee, so configuration, access control and logging still need to be set up correctly. Our guide to building HIPAA-compliant AI covers that in detail.
Prompt engineering for clinical accuracy
Generic prompts produce generic notes. A cardiology note looks very different from a psychiatry note, so we build specialty-specific templates and keep them versioned. We also pass patient context into the prompt, such as age, chief complaint and existing conditions, which improves extraction because the model no longer has to guess what the visit is about.
What good prompt design includes
- Separate templates per specialty and visit type
- Explicit instructions on what to leave out when the conversation did not cover it
- A rule that the model must not invent findings, doses or diagnoses
- Structured output so downstream systems can validate every field
EHR integration realities
FHIR R4 is the standard, but real EHR APIs are inconsistent. We have integrated with Epic, Athenahealth and several smaller systems, and each one needed custom mapping. Plan for this early. It is where most of the integration time goes, and it is rarely visible in a demo.
Accuracy and physician trust
Physicians do not trust black boxes. We built a review-and-edit interface that shows confidence by section, highlights uncertain extractions and tracks how often each section is edited. Edit rates give you a real accuracy measure instead of an impression. As the system learns from corrections, accuracy improves, and most clinicians reach 90 percent or better accuracy within two weeks of use.
Questions to settle before you build
- Which specialties launch first, and who owns their templates?
- Which EHRs must you integrate with, and who provides sandbox access?
- Where is audio stored, for how long, and who can replay it?
- How will you measure accuracy and edit rates after launch?
- What is the fallback when speech or the model is unavailable?
Building a healthcare AI product? We have done this before and can help you scope it.
Discuss your initiative




