Is ChatGPT safe for business data? Hundreds of people are reading the free version.

Somewhere right now, a contractor earning more than $50 an hour is reading a ChatGPT conversation. Not a bot. A person, with a form to fill in and a score to give. If your team uses the free version, one of those conversations could be yours.

That is the news from 404 Media’s September 14 report on “Project Lily,” OpenAI’s internal program for human review. It also settles a question small business owners have been asking since 2023: is ChatGPT safe for business data? Short version: on the Free, Plus and Pro accounts, not by default. On the Business plan, yes, and the difference costs $20 a seat. Here is the long version.

What actually happened

404 Media reported that OpenAI pays hundreds of outside contractors to read real ChatGPT conversations. The job, per the training materials the outlet saw: read the prompt, write a summary of what the person was trying to do, then rate up to four candidate replies on a scale of one to seven. They are grading tone, not facts. Less flattery, fewer emojis, less “AI-speak,” and no pretending to be a human.

The reviewers do not see your name. They do see the whole conversation, and above it they can see a “user memories” summary: what you have used ChatGPT for before, sometimes your job, sometimes roughly where you live. OpenAI runs a filter that tries to strip personal details before a chat reaches a reviewer. The company’s own line is that it tries to remove that information, and that sensitive details can still get through.

Asked where users are told that a human might read their chats, OpenAI did not answer the question directly. It sent a bulleted list saying it makes clear that human feedback improves ChatGPT and that expert raters may review conversations. After the story ran, it updated its opt-out help page. Per 404 Media, that page still does not mention human reviewers.

In plain English

Every AI company trains on some user conversations. “Training” sounds like one machine reading another. Part of it is a person reading your chat. That is the part nobody put on the sign-up screen.

Which accounts this covers matters more than the outrage. The setting is called “Improve the model for everyone,” and it is on by default for Free, Plus and Pro accounts. It is off by default for Business, Enterprise and Edu accounts, and for the API. So the $200-a-month Pro plan ships with the same default as the free one. You are paying, and you are still the product.

Two more details that matter. Turning the setting off only stops new conversations from going into the pool. Chats that were already eligible stay eligible. And deleting a conversation removes it from OpenAI’s systems within about 30 days, unless it has already been stripped of identifying details and pulled into the improvement process. At that point it is out of your hands.

Why this lands on your desk

Your risk is probably not that you personally typed something juicy. It is that somebody on your team did, on a personal account, for a good reason.

The bookkeeper pasted a payroll export to get a formula fixed. The office manager dropped in a customer’s angry email to draft a reply. Somebody uploaded a contract and asked for the short version. Every one of those is a normal Tuesday, and every one of them is now a conversation a contractor can be handed, with a summary of who that user seems to be sitting at the top. We covered how common this is in the shadow AI post: most of it happens on accounts you have never seen.

If you are in a regulated line of work, the sentence “a contractor read it” is one you do not want to say to a client, an auditor, or a regulator. If you signed an NDA, it is one you may have already broken.

The memory summary is the quiet part. The more your team leans on one personal account for work, the richer the profile above every chat. One conversation with names scrubbed is hard to identify. One conversation plus “runs a 12-person dental practice in the Midwest” is not.

What this means for you

Three rules. Do them in order.

1. Free, Plus and Pro accounts are for public information only. Nothing with a client’s name, an employee record, a bank figure, or a contract. The test: if it would be awkward in a stranger’s inbox, it does not go in the box. Same rule we gave for letting Siri read your email, because it is the same problem.

2. If your team uses ChatGPT for real work, move them onto ChatGPT Business. It is $20 per seat per month on annual billing ($25 monthly), with a two-seat minimum. Your data is not used for training by default, and you get an admin page with the switches a personal account does not have. That is the same workspace we told you to use for GPT-6 Astra. Forty dollars a month is the whole cost of making this problem go away.

3. Cannot spend the $40 yet? Flip the switch on every account today. Settings, then Data Controls, then turn off “Improve the model for everyone.” It syncs across web and mobile. Remember it is not retroactive: what was already in the pool stays in the pool. Treat it as a bandage, not a fix.

And if you are the type who runs a model at home, none of this touches you, which is the whole appeal. Here is how to do that.

The verdict

Is ChatGPT safe for business data? On Business or Enterprise: yes, about as safe as any cloud tool you already trust with your email. On Free, Plus or Pro: no, and this week we found out exactly how not.

Worth it, on the right plan. Skip it for anything confidential on the wrong one.

The opinionated bit: OpenAI could flip that default tomorrow. It has not, because the conversations are worth more to it than your goodwill. That is not a scandal, it is a business model. Plan around it the way you would plan around any vendor whose incentives are not yours.

Reading about AI is free. Falling behind on it is not. Picking the right tool for your team, putting it on the right plan, and setting the switches so nobody has to think about this again is the kind of thing BlueFort IT does for a living.