Is AI HIPAA Compliant? What Healthcare Organizations Must Know

A physician turns to ChatGPT to summarize a patient note. An office manager opens Claude to draft patient communications. A biller wonders if Gemini could help draft an appeal letter. But before using AI with protected health information (PHI), there’s one key question every healthcare professional should answer: Is this tool HIPAA compliant?

Not all AI tools are. Whether an AI application can be used in a healthcare setting depends on how it handles PHI, the safeguards it provides, and whether the vendor meets HIPAA requirements. This guide explains what HIPAA requires, where many AI tools fall short, and how healthcare providers can evaluate AI before bringing it into their workflows.

HIPAA doesn’t mention artificial intelligence by name, but its requirements still apply whenever an AI application creates, receives, maintains, or transmits PHI.

When adopting any AI tool, ask two questions:

• Will the tool have access to PHI?
• Can the vendor meet HIPAA’s legal and security requirements?

Those answers form the foundation of using AI responsibly in healthcare.

PHI includes far more than medical records. Patient names, dates of birth, diagnoses, medications, clinical notes, appointment details, insurance information, and any other data that can identify an individual in connection with their healthcare are all considered PHI.

If that information is entered into an AI application, HIPAA applies. For example, copying a patient note into an AI writing assistant to improve grammar or summarize documentation may seem harmless, but the application is now processing PHI. AI applications should be evaluated the same way you would assess any other vendor that handles patient information.

If an AI vendor creates, receives, maintains, or transmits PHI on your behalf, HIPAA generally requires a signed Business Associate Agreement (BAA).

A BAA establishes how the vendor will safeguard PHI and defines each party’s responsibilities under HIPAA. Many consumer AI tools do not offer BAAs, which means they should not be used with PHI. Without this agreement, using an AI tool with PHI can create a HIPAA compliance issue regardless of how useful or secure the application appears to be.

Thousands of AI tools are now available, but most were designed for general business or personal use, not healthcare. As a result, many lack the safeguards healthcare organizations need to protect patient information and meet HIPAA requirements.

Free and consumer-grade AI applications usually aren’t built for handling PHI. Common gaps include:

• No BAA available
• Data may be retained or used for model training
• Limited audit logging
• Minimal access controls
• Little transparency around data storage and retention

These aren’t minor gaps that can be solved by changing a privacy setting or opting out of model training. They’re fundamental compliance requirements that should be evaluated before introducing any AI tool into a workflow involving patient information.

Many healthcare groups start using AI to improve efficiency, not realizing they’ve introduced new compliance risks. When a department downloads a new tool or an employee experiments with an AI assistant without proper oversight, seemingly harmless decisions can create HIPAA exposure.

Here are some of the common ways healthcare providers inadvertently create HIPAA exposure when adopting AI tools.

Using an AI tool that processes PHI without a signed BAA is one of the most common compliance mistakes. This often happens when an employee uses AI to summarize a patient note, rewrite a clinical document, or draft a patient communication without realizing PHI has been shared with a third party. Even a single incident can create serious HIPAA risk.

The HIPAA Security Rule requires covered entities and business associates to conduct an ongoing risk analysis of systems that create, receive, maintain, or transmit electronic protected health information (ePHI). AI tools should be included in that process.

Your HIPAA risk assessment should identify what PHI an AI tool accesses, how the data is protected, and whether appropriate safeguards are in place. As AI adoption grows, regularly review your technology inventory to identify unapproved AI applications already being used.

HIPAA requires more than protecting PHI. You also need to be able to demonstrate who accessed it and when.

Many AI tools lack role-based permissions, detailed audit logs, or centralized administrative controls. If your organization can’t demonstrate how PHI is being accessed within an AI application, that gap may become apparent during an audit or investigation.

Many AI platforms retain customer inputs or use them to improve their models unless specific contractual and technical safeguards are in place.

Before using an AI tool with PHI, understand whether data is retained, whether it is used for model training, and how those practices are documented. If a vendor uses PHI for purposes beyond providing its service, it creates significant HIPAA risk.

Some AI vendors have built products specifically for healthcare, but there is no government certification that makes an AI tool “HIPAA compliant.”

Instead, evaluate whether the vendor supports HIPAA requirements. At a minimum, look for:

• A signed BAA
• Encryption of PHI in transit and at rest
• Role-based access controls
• Audit logging
• Clear data retention policies
• Contractual assurance that PHI will not be used to train AI models

Healthcare-specific AI platforms are often a safer starting point than general-purpose tools, but every vendor should be evaluated through your risk assessment and vendor review process prior to implementation.

Using AI in healthcare isn’t just about choosing the right vendor. You also need the policies, processes, and training to ensure AI is used appropriately. Before rolling out any AI tool that may interact with PHI, focus on three key steps.

Evaluate every AI tool that handles PHI before it’s deployed.

Your assessment should identify what PHI the tool will access, how it protects that information, whether the vendor offers a BAA, and whether any security gaps exist. Documenting those findings is an important part of maintaining an effective HIPAA compliance program. Compliancy Group can help you build a risk assessment process that accounts for AI tools alongside the rest of your HIPAA program.

Develop written policies that define how AI may be used with PHI.

Those policies should identify approved AI tools, prohibit the use of consumer AI with patient information, establish data handling requirements, and explain how employees should report potential AI-related privacy or security incidents.

Ensure employees understand which AI tools are approved, when PHI can be used with AI, and when it cannot.

AI guidance should become part of workforce HIPAA training, since workforce training is one of the OIG Seven Elements of an Effective Compliance Program and a direct HIPAA requirement. As with any compliance policy, employees are far more likely to follow expectations when they understand both the rules and the reasons behind them.

AI offers endless opportunities to improve efficiency across healthcare, but it doesn’t change an organization’s HIPAA responsibilities.

When introducing any AI application that may interact with PHI, evaluate the vendor, complete a risk assessment, establish clear policies, and train your workforce on appropriate use.

Compliancy Group helps healthcare professionals put those safeguards into practice. Whether you’re conducting HIPAA risk assessments, managing policies and procedures, training employees, tracking vendors, or documenting compliance activities, The Guard provides a centralized platform to support the ongoing oversight that HIPAA requires, even as new technologies like AI become part of everyday healthcare.