If you are a medtech founder chances are you are considering “using AI”. But it can be hard to know where to start and what good looks like. Teams jump to a model before they define a care gap. They ship a glossy demo no one trusts. They add clicks to already stretched workflows. Procurement raises information governance concerns. Momentum stalls and the work never reaches a paid contract.
Eric Topol’s Deep Medicine is a useful reset. It helps you answer three practical questions. Where can AI add real clinical value. What “good AI in care” looks like. How to design something clinicians will actually use and healthcare buyers will take seriously. This week’s bookclub turns Topol’s big ideas into steps you can apply this week.
The core message of Deep Medicine
Topol’s thesis is clear. AI should augment clinicians, not replace them. The goal is more human care by giving clinicians back time and better tools.
“The greatest opportunity offered by AI is not reducing errors or workloads, or even curing cancer. It is the opportunity to restore the precious and time-honored connection and trust, the human touch, between patients and doctors.”
Where AI can help most when done well:
- Diagnostics and pattern recognition. Finding signals in imaging, waveforms, pathology and genomics faster and more consistently than humans.
- Workflow support. Cutting repetitive admin so clinicians can focus on patients.
- Personalised care. Using data to tailor decisions for the individual.
There are conditions. Systems must be explainable, accountable and safe. Data quality, bias control, privacy and real workflow integration are non-negotiable. Black-box predictions that add friction will not earn trust. Topol’s interviews and commentary underline this human-first lens. AI can support doctors, not replace them, especially where context and judgement matter.
Key lessons for AI founders
- Start with a care gap, not an algorithm. Define the clinical or operational job to be done. Who is in pain today. How do you relieve that pain next week, not in theory.
- Clarity over mystery. Make the reasoning and the output understandable at the point of care. If a clinician cannot see why the system thinks what it thinks, adoption will stall.
- Proof beats promises. Plan early for evaluation that matches NHS expectations. Align with NICE’s Evidence Standards Framework so your claims about outcomes, safety and cost stand up in scrutiny. NICE
- Fit the work. Observe clinics and wards. Remove steps. Avoid context switching. If your tool adds clicks, it will die on the ward.
- Build ethics in from day one. Address bias, privacy, data rights and accountability. Keep an audit trail. Be ready to show what you do when the model is uncertain or wrong.
- Keep a human in the loop. Design for appropriate oversight and graceful failure. Support judgement rather than replacing it.
The NHS layer: what “good” looks like under UK rules
Regulation. If your product performs a medical function, assume it is Software as a Medical Device. You will need the right regulatory cover such as UKCA marking and documentation that matches your risk class. MHRA’s software and AI guidance explains how software is regulated in the UK and what evidence you need. Build your technical file and change control now rather than later. Useful overview here: GOV.UK
Clinical safety. NHS England’s digital clinical safety standards set clear requirements.
- DCB0129 applies to manufacturers. You are expected to create and maintain a Clinical Safety Case, appoint a Clinical Safety Officer, and run a live hazard log through the lifecycle. NHS England Digital
- DCB0160 applies to healthcare organisations that deploy your system. They must evidence safe local use, but you will be asked for supplier artefacts to support that process. These standards are being reviewed to keep pace with new tech. Keep an eye on updates. NHS England Digital
Evidence. NICE’s Evidence Standards Framework sets proportionate expectations for digital tools. Map your claims to the right tier. Plan a prospective evaluation with a partner trust that measures safety events, clinical outcomes, time saved and net cost. Treat this as part of product development, not an afterthought. NICE
Information governance. If your processing is likely high risk, complete a Data Protection Impact Assessment. Use the ICO’s guidance and checklists. Address legal basis, data minimisation, transparency and security. Expect buyers to ask for your DPIA and your data flow map. ICO
Operational readiness. NHS buyers will look for audit trails, performance monitoring, bias checks, model drift monitoring and a clear rollback plan. Put these in your safety case and SOPs. They help clinicians trust the system and help your team respond fast when things change.
Mini-case: an AI triage tool that is safe, useful and credible
Scenario. A startup wants to support urgent care triage by predicting risk of hospital admission within 72 hours. The aim is to help clinicians prioritise assessment and follow up.
Design for trust. The interface shows the risk score and an explanation in plain English. It highlights the top factors that drove the score. It shows an uncertainty range. It offers quick links to relevant guideline prompts. When confidence is low, it defaults to clinician judgement and recommends standard pathways.
Safety artefacts. The team maintains a DCB0129 Safety Case with a live hazard log. Examples include misclassification risk, over-reliance risk, and bias against groups with sparse data. Each hazard has mitigations, tests and monitoring. A named Clinical Safety Officer signs off releases.
Evaluation plan. The company runs a 12-week stepped-wedge service evaluation across three sites in one ICS. Primary metrics: triage decision time, revisit rate within 7 days, admissions captured at appropriate acuity, safety events. Secondary metrics: clinician satisfaction, training time, net cost per patient. As a worked example, if the tool saves 90 seconds per consult, a site seeing 160 patients per day frees about 4 hours of clinician time daily for higher value care. Labelled as modelled impact, then tested in the study.
IG and deployment. A DPIA is completed and shared with the Trust’s IG team. Data flows are mapped, with retention limits and clear patient information. Audit logs capture inputs and outputs for each decision. The product has a rollback plan and a support model with defined response times.
Procurement readiness. The team can supply the UKCA plan, Safety Case summary, evaluation protocol, and a reference site. A clinical champion offers a quote about time saved and confidence in use. The pitch frames the tool as a way to help clinicians focus on patients, not as a replacement for clinical judgement.
Pitfalls to avoid
- Selling doctorless care. This triggers scepticism. Frame your product as clinician support and show where human judgement sits. Topol’s central argument is human first.
- Leaving IG to procurement. A missing DPIA will stall you. Draft it early and share a plain-English summary with buyers. =
- Overfitting to one site. Designing only for one trust’s workflow makes scaling hard. Capture variants during discovery. Build configuration rather than one-off custom code.
- No rollback plan. Buyers will ask what happens if the model degrades or a data source fails. Define thresholds, alerts and a manual override path.
- “It is proprietary” when asked about reasoning. You can protect IP and still provide an interpretable layer that clinicians can understand at the bedside.
- Evidence that does not match claims. If you claim time saved, measure time saved. If you claim better safety, measure safety events. Align with the ESF. NICE
Toolkit: 6-point founder checklist
- Reasoning and outputs. Can a clinician understand why and what the system recommends in plain English at the point of care.
- Regulatory map. Do you know your SaMD class, your MHRA route and your UKCA plan? Have you started the technical documentation?
- Clinical safety. Do you have a draft DCB0129 Safety Case, a named Clinical Safety Officer and a live hazard log?
- Workflow fit. Have you observed real clinics, removed steps and tested a paper or clickable prototype with end users?
- Evidence plan. Is there an agreed evaluation with a partner trust aligned to NICE’s Evidence Standards Framework, with success metrics and sample size?
- Governance. Do you have a DPIA, audit trails, bias monitoring and a rollback plan for model drift or outages?
Want more like this?
If you found this useful, join the Medtech Mentor newsletter and get our free starter pack for building NHS-ready products. Sign up below. You will also get updates on our Bookclub reviews so you can learn from the greatest minds in Medtech and startup.
While you are here, check out our other Bookclub reviews on our blog and our practical guides on NHS routes to market, information governance and clinical safety. They are filled with practical advice, tools and tips to help you on your entrepreneurial journey.
Get your copy of Deep Medicine here, its a must read for anyone thinking about using AI in medtech!
Thanks for reading!!
The Medtech Mentor Team
P.S. we do use affiliate links which means we get a small amount of money if you buy something – this is at no cost to you and we never recommend anything we don’t think is awesome! Thanks for reading!