Where does your data end up when you use AI?
When an employee pastes text into an AI service, the text leaves the organisation. Three questions you should be able to answer, two documented examples and five steps to get an overview.
When an employee pastes text into an AI service, that text leaves the organisation. Where it is sent, how long it is stored and who can demand to see it is decided by the agreement with the provider, not by the person who presses “send”.
This is not a theoretical problem. In IBM’s 2026 Cost of a Data Breach Report, more than 20% of organisations reported a breach targeting AI models or applications. The most common causes IBM points to are compromised APIs, applications or plug-ins (27%) and cloud misconfigurations affecting AI workloads (27%). The average data breach cost USD 4.99 million, the highest IBM has recorded (IBM). The figures are based on breaches at 602 organisations between March 2025 and February 2026.
Below are three questions you should be able to answer for every AI service you use, two examples of what happens when data goes astray, and five concrete steps to get an overview.
1. Where is the data sent, and what is it used for?
The same service can have very different rules depending on whether an employee uses a personal account or the organisation’s agreement. Two examples:
- Anthropic (the maker of Claude) changed the terms for its consumer plans Free, Pro and Max in August 2025. If the user turns the setting on, conversations are used to train new models and are retained for five years. Otherwise, 30-day retention applies. The change does not apply to the business services or the API (Anthropic).
- OpenAI states that it does not train models on business data by default, while consumer users control in their settings whether their chats are used to improve ChatGPT (OpenAI).
An employee who uses their own free account for work text is therefore covered by the consumer terms, not by the agreement the organisation has signed.
2. Where is the data stored, and for how long?
Deleting something in the app does not necessarily mean it has been deleted by the provider.
In 2025, a US court ordered OpenAI to retain content from ChatGPT and the API indefinitely, including conversations users had deleted themselves. The order came in the copyright lawsuit The New York Times has brought against the company. It covered ChatGPT Free, Plus, Pro and Team, and API customers without a Zero Data Retention agreement. ChatGPT Enterprise, ChatGPT Edu and API customers with such an agreement were not covered (OpenAI).
The obligation ended on 26 September 2025. OpenAI says it still retains a limited set of user data from April to September 2025 because The New York Times demands it, but that it is no longer required to retain conversations from the EEA, Switzerland or the United Kingdom.
Two things are worth noting. The type of agreement determined whether the data was covered. And a court in another country can order the provider to keep data longer than its terms say.
3. Who can demand to see the data?
US law (the CLOUD Act) requires providers of communication and cloud services to preserve or disclose data in their possession, custody or control, “regardless of whether such communication, record, or other information is located within or outside of the United States” (18 U.S.C. § 2713).
At a hearing in the French Senate in June 2025, Anton Carniaux, director of public and legal affairs at Microsoft France, was asked whether he could guarantee under oath that data on French citizens could not be transmitted to the US government without the explicit agreement of the French government. His answer, as reported by The Register: “No, I cannot guarantee that, but, again, it has never happened before.” He added that Microsoft rejects requests that are unfounded, and that no European company or public body has been affected since Microsoft began publishing transparency reports (The Register).
At the same time, transferring personal data to the US is lawful today. Personal data may only be transferred out of the EEA when the conditions in Chapter V of the GDPR are met (Article 44), and for the US, organisations can rely on the European Commission’s adequacy decision. On 3 September 2025, the EU General Court dismissed an action seeking to have the decision declared invalid, and the Norwegian Data Protection Authority (Datatilsynet) wrote the same day that organisations can continue to rely on it. Its director of legal affairs, Erlend A. Methi, also said: “For several reasons, this judgment is hardly the final word. Our recommendation to think about preparedness and to have a strategy for alternative solutions still applies” (Datatilsynet, in Norwegian; our translation). The judgment has been appealed, and the appeal is listed as pending before the Court of Justice of the EU as of 17 November 2026 (Court of Justice of the EU, case C-703/25 P).
One does not rule out the other. A transfer can be lawful while the provider can still be required to hand over data under US law.
When data goes astray: two examples
Samsung: source code in ChatGPT
In April 2023, Samsung engineers uploaded internal source code to ChatGPT, according to an internal memo seen by Bloomberg (CSO Online). Samsung explained its concern by saying that data sent to such services is stored on external servers, is difficult to “retrieve and delete”, and could be disclosed to other users (TechCrunch).
From 1 May 2023, Samsung introduced a temporary restriction on generative AI on company computers, tablets and phones. Employees were asked not to enter company information or personal data into such tools on other devices. In an internal survey, about 65% said that using generative AI tools carries a security risk (TechCrunch).
It is not known whether the code went any further. The memo did not say whether anyone outside Samsung had seen it (CSO Online). But by the company’s own assessment, the code was stored somewhere it was difficult to retrieve from, and Samsung chose to stop a tool that one of its divisions had been allowed to use since 11 March the same year (TechCrunch).
Salesloft Drift: when the chatbot holds the keys to customer data
Drift is a chatbot service from Salesloft that many organisations had connected to their Salesforce CRM. From as early as 8 August until at least 18 August 2025, a threat actor used stolen access tokens (OAuth tokens) from the Drift integration to export large volumes of data from numerous organisations’ Salesforce instances. The actor then searched the data for secrets such as AWS access keys, passwords and Snowflake access tokens. The flaw was not in Salesforce itself. On 20 August, Salesloft and Salesforce revoked all active tokens for Drift (Google Threat Intelligence).
Cloudflare was among those affected. The attackers obtained text and contact information from Cloudflare’s support cases. Cloudflare found 104 Cloudflare API tokens in the stolen data and rotated all of them. The company notified all affected customers directly and asked them to consider anything they had pasted into a support case compromised. Cloudflare’s own services and infrastructure were not compromised (Cloudflare).
Cloudflare’s customers were thus affected through an integration at a supplier’s supplier. Google subsequently advised all Drift customers to treat all tokens stored in or connected to Drift as potentially compromised (Google Threat Intelligence). Every integration is a key. The question is what it opens.
How to get an overview
- Make a list of the AI services actually in use, and which version: personal account or business agreement. As the examples above show, the terms follow the version.
- Require a data processing agreement. The GDPR requires a processor to be bound by a written contract (or other legal act), and not to engage another processor “without prior specific or general written authorisation of the controller” (Article 28). Ask for an overview of sub-processors, where the data is stored and for how long.
- Map the data flow for each purpose. In its regulatory sandbox project with NTNU on Microsoft 365 Copilot, Datatilsynet recommends mapping the processing “from the moment an instruction is given to the copilot until the answer comes out”, and says the data protection impact assessment (DPIA) must be carried out “per processing operation or set of similar processing operations” (Datatilsynet, in Norwegian; our translation; Article 35). The authority also points out that the copilot “will have access to all the same information as the user of the tool has”. Its conclusion is that Norwegian organisations can adopt the tool, “but not for everyone and not for everything”. The advice from the sandbox is guidance, not a formal decision.
- Give integrations as little access as possible. After the Drift attack, Google recommended reviewing all third-party integrations, revoking and rotating credentials, and restricting what connected apps can access (Google Threat Intelligence).
- Have a plan for incidents. A personal data breach must be notified to the supervisory authority (in Norway, Datatilsynet) “without undue delay and, where feasible, not later than 72 hours after having become aware of it”, unless it is unlikely to result in a risk to the rights and freedoms of natural persons. A provider acting as your processor must in turn notify you “without undue delay” (Article 33).
How we do it
We believe safety is documentation, not promises, and that you should ask us the same questions you ask other providers. Our answers are on our Safe AI page. The solutions we build run on our own servers in Sandefjord by default. Every customer gets a data processing agreement, and where the data is stored is agreed and documented per project. Where it gives the best result, we also use the best models available. An open overview of which models are used where is something we are working towards. It is not finished yet.
Further reading
- What Copilot can see in your organisation, and what to tidy up before launch
- LLM Security: Best Practices for Large Language Models in 2026
- GDPR and AI: Legal Considerations for Norwegian Businesses
Sources
All sources were read and checked on 17 November 2026.
- IBM, “IBM Study: One in Four Malicious Breaches are AI-Enabled, Costing Companies $6 Million on Average”, 29 July 2026
- IBM, Cost of a Data Breach Report 2026 (overview page)
- Anthropic, “Updates to Consumer Terms and Privacy Policy”, 28 August 2025
- OpenAI, “How we’re responding to The New York Times’ data demands in order to protect user privacy”, 5 June 2025, updated 22 October 2025
- CLOUD Act, 18 U.S.C. § 2713 (Cornell Law School)
- The Register, “Microsoft admits it ‘cannot guarantee’ data sovereignty”, 25 July 2025
- Datatilsynet, “EU-domstolen opprettholder adekvansbeslutningen for USA”, 3 September 2025 (in Norwegian)
- Court of Justice of the EU, case register for C-703/25 P Latombe v Commission (status “Pending” as of 17 November 2026)
- CSO Online, “Samsung bans staff AI use over data leak concerns”, 2 May 2023
- TechCrunch, “Samsung bans use of generative AI tools like ChatGPT after April internal data leak”, 2 May 2023
- Bloomberg (via The Star), “Samsung bans staff’s AI use after spotting ChatGPT data leak”, 2 May 2023, partly paywalled
- Google Threat Intelligence Group, “Widespread Data Theft Targets Salesforce Instances via Salesloft Drift”, 26 August 2025, updated 28 August 2025
- Cloudflare, “The impact of the Salesloft Drift breach on Cloudflare and our customers”, 2 September 2025
- Datatilsynet, “NTNU: Copilot med personvernbriller på”, 26 November 2024 (in Norwegian)
- General Data Protection Regulation (EU) 2016/679 on EUR-Lex: Article 28, Article 33, Article 35 and Article 44
- Skjld Labs, “Safe AI”