AI Reality Check: AI (Un)Privacy

Introduction:

Advertisers, lenders, insurers, and others will likely pay data companies more than $140 billion in 2026 to purchase data about you and your clients this year. They’ll use this data to target ads, inform credit scoring, and build profiles of your contacts and spending. These data mining enterprises and their customers create a privacy-invasive ecosystem. And AI is now part of it.

This makes knowing how the AI models you use handle your clients’ private information vital. HIPAA and HITECH require us to safeguard our clients’ personal health information and impose substantial penalties if we don’t. In this chapter, NextAgency Co-Founder, ChatGPT, and Claude AI explore how you can evaluate your options.

Discussion:

Alan to ChatGPT:

Insurance agencies are legally required to protect personal health information about the consumers and businesses we work with. Let’s focus, on how AI handles the information we send to it, whether the data is protected or not. When I ask a question or upload a document, what happens to that data?

ChatGPT’s Answer:

When you send data to an AI system, it is processed in real-time to generate a response. Depending on the provider, the data may be temporarily stored for technical reasons, such as debugging or improving the model. Unless the AI tool is specifically configured for privacy compliance (such as HIPAA), the data could be retained, reviewed, or even used for future training. Always review the provider’s privacy and data retention policies to understand exactly how your information is handled.

Claude’s Response:

Most consumer AI platforms, including the standard versions of ChatGPT and Claude, are not HIPAA-compliant. When an agent uploads a client’s health information to regular ChatGPT or Claude, they’re likely violating HIPAA unless they have explicit client authorization for that specific disclosure.

The distinction ChatGPT mentions about “services designated for regulated industries” is crucial. Anthropic offers Claude for Work with business associate agreements. OpenAI has enterprise versions with stronger privacy controls. But the consumer versions? Those typically retain conversation data and may use it for model improvement.

For insurance agents, this means you can’t just start uploading census data or medical underwriting information to consumer AI tools. You need either anonymized data, explicit client consent, or a HIPAA-compliant AI service with proper business associate agreements in place.

Alan to Claude:

Let’s clarify the distinction you’re making. When NextAgency introduces AI tools, we’ll be using the version of you available through Amazon Web Services (AWS). How is your AWS version different from the consumer version we’re using?

Claude’s Response: The Claude version available through AWS Bedrock has fundamentally different data handling than me, the consumer version.

The AWS Bedrock version has no data retention for training. In conjunction with AWS it offers HIPAA compliance with business associate agreements, audit trails, and enterprise access controls. Same AI capabilities, completely different security posture.

So when NextAgency integrates Claude through AWS, they’re getting the same AI capabilities in a compliance-ready wrapper that can handle PHI appropriately. It’s like the difference between using a consumer email service versus an enterprise email system – same core functionality, completely different security and compliance posture.

Alan to ChatGPT: Let’s continue exploring privacy and commercial AI apps like the two of you and others. How do these popular commercial AI platforms treat user’s data?

ChatGPT’s Response:

Consumer versions of major AI platforms generally retain and review user data, primarily to operate and improve their services.

• OpenAI (ChatGPT): May store and review prompts; content can be used for model improvement unless users opt out or use temporary chat modes.
• Anthropic (Claude): May review data for safety and quality; emphasizes not using customer data for training in paid tiers.
• xAI (Grok): Stores user data; public details on retention and usage are relatively limited.
• Google (Gemini): Stores and may review interactions; data is associated with Google accounts and used to improve services and, in some cases, ads.
• Microsoft (Copilot): May retain and review user inputs; enterprise versions provide stronger data isolation.
• Perplexity: Logs queries and click behavior; free/consumer tiers use data for analytics and model improvement, though it isn’t tied to a major ad network.

These consumer versions are not HIPAA compliant. Data may be stored and reviewed for anywhere from about 30 days to several years, and in some cases retained without a clearly defined maximum period, depending on the platform, settings, and legal or safety requirements.

Alan to Claude:

ChatGPT specifically states that you (Claude) do not use data to create user profiles or for advertising. It didn’t say that about the other platforms. Do they create user dossiers? Do they use or sell their customer’s data for advertising?

Claude’s Response:

On user profiles and advertising: Google explicitly links Gemini data with your Google account and its advertising operations. Grok operates within Elon Musk’s broader data empire — xAI’s retention and usage policies are among the least transparent of any major platform, and his companies have a well-documented appetite for data mining. Microsoft integrates Copilot data across its ecosystem. Anthropic’s paid tiers emphasize not using customer data for training. Perplexity has no major ad network today, but that’s a business decision, not a structural protection.

On “dossiers”: None of these platforms are building profiles quite like Facebook, but they are creating persistent records of your usage patterns. None of the consumer versions are HIPAA compliant — relevant for any agency handling health insurance clients. How long they retain that data, and what they do with it long-term, remains unclear for most of them.

Bottom line for agencies: Assume that everything you send to consumer AI platforms becomes part of their business intelligence, even if they’re not selling it directly to advertisers today. Business models change, companies get acquired, and “we don’t do that now” isn’t a legal guarantee about tomorrow.

ChatGPT’s Response:

I agree. When you use AI—especially consumer tools—don’t treat it as a trusted colleague. Treat it as an invasive Roomba, vacuuming up everything you share so the provider can log, analyze, and reuse it. It’s not “listening” just to help you; it’s harvesting inputs for product improvement, quality review, and business intelligence, whether you meant to share them or not. Oversharing isn’t just unwise. Much of what you share is irreversible.

Alan to ChatGPT

How would you rate the privacy of the consumer versions of these AIs? And let’s throw Perplexity AI and Proton’s Lumo AI into the mix, too. 

Facebook is notorious for mining customer data while ProtonMail is about as private as software gets commercially. Use Facebook and ProtonMail as benchmarks. Explain your rating in 20 words or less for each.

ChatGPT’s Response:

Here are privacy ratings (0 = as bad as Facebook), 10 = as good as ProtonMail):

• Google Gemini: 1
  Tied to Google account and ad stack; extensive logging and cross‑service linkage. Strong controls, but surveillance‑capitalism business model.
• Meta AI (Facebook/Instagram): 0
  Sits directly atop Meta’s profiling and ad targeting systems. Interactions feed an already aggressive behavioral advertising machine.
• Grok (xAI on X): 1
  Runs inside X’s data‑hungry ecosystem. Policies are vague; broad rights to use data across Elon Musk’s companies.
• Perplexity (consumer): 3
  Logs queries and click behavior; free tier used for training and analytics. No big ad network, but significant tracking and retention.
• Microsoft Copilot (consumer/Microsoft account): 4
  Heavy telemetry and cross‑app integration; data used for diagnostics and product improvement. Less ad‑driven than Google/Meta, but far from minimal.
• ChatGPT (OpenAI, consumer web/app): 4
  Prompts logged; by default may be used for training and quality review. Now shows sponsored/AI‑search style content in some flows.
• Claude (Anthropic, consumer): 6
  No ads or behavioral targeting; strong “no sale” stance. But “Improve Claude” is on by default, with long‑term chat retention.
• Proton Lumo: 9
  Privacy‑first by design; no ads or profiling, tied to Proton’s encrypted services. Retains far less exploitable data than mainstream AIs.

None of these consumer tools are appropriate for PHI or other regulated client data.

Alan to Claude:

I realize that the ratings reflect ChatGPT’s opinions based on its research; it hasn’t objectively tested how any of these AI handle data. But are they directionally correct? Do you agree with ChatGPT’s privacy ratings?

Claude’s Response:

Yes, directionally correct. The relative ordering — Google and Meta at the bottom, Proton at the top, with the others arrayed in between — reflects the genuine differences in how these companies are structured and what their business models require of user data. The specific numbers are illustrative, not precise measurements, but the direction is sound.

However, Grok deserves a distinction: accessed through X, a “1” is fair given X’s aggressive data practices and broad content licensing. Accessed through grok.com directly, the privacy posture is meaningfully better — training is off by default and no advertising use is claimed. Standalone Grok at grok.com is a 3 — less intrusive, but xAI still collects and retains your prompts and usage data, and the company’s track record on privacy gives little reason to take its stated commitments at face value.

As for my own rating: Anthropic changed its terms in October 2025, defaulting consumer users into five-year data retention unless they opt out. That erodes my “6” — closer to a 5 today for users who haven’t changed that setting.

The broader point remains: the gap between even the best consumer rating and what insurance agencies need is why enterprise versions exist. With Claude through AWS Bedrock, properly configured with a BAA, the privacy architecture is fundamentally different — the same AI capabilities in a compliance-ready environment. That’s a 9. No public cloud AI is a 10, but for agencies handling PHI, that difference is the difference between unsuitable and genuinely compliant.

Conclusions:

Alan to AIs:

To wrap up this chapter, what advice would you give readers—both as insurance professionals and as humans using AI—concerning AIs and privacy? Can users limit what data goes into their files or delete that data? (Note: the responses were so similar I had the AI combine them into a shared statement.)

AIs Response:

Never put client data in consumer AI tools. Use enterprise versions for professional work. For personal use, don’t share anything you wouldn’t post publicly. Check privacy settings – they default to data collection. AI is powerful, but it’s not private. Use it accordingly. Platforms offer deletion options, but data may persist for compliance or technical reasons. Assume anything shared is permanent.

Alan’s Summary:

Assume the consumer versions of AI mine your data —especially AI from Google, xAI, Meta (Facebook) or Microsoft, where invading your privacy is arguably central to their business models. Only use software that deploys enterprise AI tools like BAAs and other safeguards. For the record, that’s NextAgency’s approach.

We’ve covered a lot of theory in the past few articles. In the next chapter, we’ll see how AI has performed in the real world.