A couple of months ago, the rise of AI tooling was really taking off. Many software engineers began publicly admitting to not writing very much code anymore. In my own office, many employees were utilizing a litany of AI tools, and I spun up a directive to investigate the new arrival of enterprise AI tooling for small organizations.
For my medical practice of roughly 30 people, based on any current offering, flagship AI chat subscription would run easily $750/month. After running some quick estimations and having a few conversations with the provider and admin staff, I was able to determine that each admin user was using between 3-5 random free AI tools at any given time. Providers tended to do more research focused queries, and invest significant time and personal funding into 1-2 main tools.
As far as what the market has to offer, there are some compelling products for medicine. Our main tool cost nearly $120/month per seat for “Clinical AI” trained on pubmed data. Another vendor focusing more on defense but branching into healthcare charged $225/month for 1 million HIPAA compliant tokens in/out. Useful in practice, and one of the early adopters to take advantage of AWS Bedrock style offerings from major cloud providers. More importantly for us, models under AWS Bedrock around this time also came under the AWS BAA, removing the primary legal blocker for clinical use.
Second, all the random AI tools have to be cold started every time, and it leads to clinical messaging and branding being off, ugly, or plain incorrect. This is a case where spending 2-3 weeks getting an MVP live on AWS bedrock with clinical branding, context, guardrails, and clear PHI controls constitutes as 100x times savings at the time of this writing, from 30 seats * $25.00 or $750 per seat to $8.63 in AWS fees, for the entire practice, in its first month. Most days has about 75-150 queries, 80% of which automatically routed to the smallest and cheapest model. I am sure this will raise to the mid $30s before too long, but even with rapid enshittification happening among major AI vendors, a quadrupling in price would be nominal.
The most interesting part of this transfer is that even though most users are on free other AI plans, their relationships with the tones, styles, etc of their individual AI chats introduce friction to using the clinical tool. Despite being maybe 3-4 weeks behind major AI vendors with features, that tiny gap is all it takes for people to feel more comfortable with the flagship offerings.
The architecture relies on our identity provider > AWS > a very generic looking AI chat UI that has a few practice specific features to encourage towards haiku for speedy edits, sonnet for document summarization, and opus for deep research, as well as shared / siloed functionality with a robust PHI scrubber module and DynamoDB tables tracking all activity and queries. Additionally, full admin controls, including budgeting per user, adding / disabling tool usage.
In the clamor for IPOs and market differentiation, everyone is fighting for institutional knowledge and walled gardens of their own. What has been most helpful about the clinical AI is the document ingestion of all clinical practices, protocols, and a mechanism for journal article ingestion and approval by the chief medical director and COO, as well as removing multiple surfaces for searching of internal documents. I would not be surprised to witness a struggle between wholly owned vector databases per business or individual, or closed knowledgebases locked behind corporate vendor contracts.
Assuming the practice ever gets protocols flushed out to the point that large models are not necessary for document retrieval / summarization, there is a conversation to be had about fully self hosting an open source model in the 70b parameter range for the majority of back office tasks.
P.S. Don’t tell my CEO, but its not running on an MCP server, or even vector database yet, as the corpus of clinical documents is not large enough to warrant chunked indexing.