# AI Assistants on Your Own Data: RAG and Privacy in Practice

How AI assistants built on your company data work, and what to consider for GDPR, provider choice and the EU AI Act, with a checklist and a pilot plan.

by Alexander Herling Published 27 October 2026 8 min read

![Abstract illustration: stacked light blue document sheets with lines leading to a deep blue speech bubble, enclosed by a protective ring](https://degoya.digital/blog/ai-assistants.svg)

A chatbot that answers questions about your products, manuals or internal policies is achievable today with a reasonable amount of effort. The technique behind it is usually Retrieval-Augmented Generation, or RAG. The real questions are rarely technical but about data: which documents may the assistant use, who may see which answers, where is data processed, and what do the GDPR and the EU AI Act require? This article explains the basics and shows how to start safely with a small pilot. It focuses on EU rules, which apply whenever you process personal data of people in the EU.

**Note:** This article is a general overview and not legal advice. For a legal assessment of your project, please involve your data protection officer and, where needed, a lawyer.

## How RAG works

A language model does not know your company documents. RAG hands it the relevant passages with every question. The process in five steps:

1. **Collect documents:** manuals, FAQs, product data, wiki pages, policies.
2. **Split them into chunks:** long documents are broken into short, meaningful passages, for example by heading.
3. **Compute embeddings:** each passage is turned into a numeric vector that represents its meaning and stored in a vector database.
4. **Retrieve matching passages:** when someone asks a question, the system finds the passages closest in meaning, often combined with classic keyword search.
5. **Answer with sources:** the language model writes an answer from those passages and points to the documents it used.

### RAG or fine-tuning?

Fine-tuning retrains a model on your own examples. That works for improving tone, format or specialised tasks, but hardly for reliably retrieving current knowledge.

| Criterion        | RAG                                 | Fine-tuning                                 |
| ---------------- | ----------------------------------- | ------------------------------------------- |
| Freshness        | New document, available immediately | Retraining required                         |
| Source citations | Possible                            | Not possible                                |
| Access rights    | Filterable per user                 | Knowledge sits in the model, not filterable |
| Deletion         | Remove the document                 | In practice only by retraining              |
| Good for         | Knowledge, documentation, support   | Style, format, specialised tasks            |

For most company assistants RAG is the right foundation, also for data protection.

## Data sources and permissions

The most common mistake in RAG projects: every document goes into one shared index, and every user can query everything. Suddenly the assistant tells a trainee about salaries or contract terms.

Ground rules:

- **Approved sources only.** Decide which systems and folders are connected. Everything else stays out.
- **Carry permissions through.** Each passage inherits the permissions of its source document. Retrieval filters by the asking user's rights before the model sees anything.
- **Separate public and internal.** A chatbot on your website only accesses public content. Internal assistants run separately and require sign-in.
- **Avoid personal data** where the purpose does not need it. A product assistant does not need your customer database.
- **Keep the index in sync.** Changed, deleted or restricted documents must be reflected promptly.

## GDPR: the key points

As soon as personal data is involved, in the documents, in users' questions or in the logs, the GDPR applies.

**Legal basis.** Internal assistants can often rely on legitimate interests or the employment relationship; customer service on contract performance or legitimate interests. Which basis holds depends on the specific purpose and should be documented. Where works councils exist, as in many German companies, they usually need to be involved for tools used by employees.

**Processing agreements.** If you use external model, hosting or database providers, you need a data processing agreement under Article 28 GDPR with each of them. Check the list of sub-processors too.

**Transfers outside the EU.** For transfers to the US, the EU-US Data Privacy Framework can serve as the basis, provided the recipient is certified. The EU General Court upheld the adequacy decision in September 2025, but an appeal is pending before the Court of Justice. If you want extra certainty, add standard contractual clauses or choose processing within the EU.

**Data minimisation.** Index only what the assistant needs. Replace names and identifiers in sample data where possible.

**Retention and deletion.** Decide how long conversation histories are kept and delete them automatically afterwards. Erasure requests must also reach the index and the logs.

**Logging.** Logs help with quality assurance and abuse detection but often contain personal data. Log selectively, protect the logs and limit retention.

**Data protection impact assessment.** With new technologies, large-scale processing or sensitive data, a DPIA under Article 35 GDPR may be required. For internal assistants with access to HR or health data, expect to need one. Your data protection function makes the call.

**Transparency.** Your privacy notice describes what data the assistant processes, for what purpose and with which providers.

## Choosing a provider

There are three basic options:

| Option                                     | Advantages                                        | Watch out for                                              |
| ------------------------------------------ | ------------------------------------------------- | ---------------------------------------------------------- |
| Model API from a major provider            | Strong models, quick start                        | Check contract, location, retention, training              |
| Model via a cloud platform in an EU region | Processing in the EU, existing contracts reusable | Model availability varies by region                        |
| Open-weight model, self-hosted             | Full control over data and location               | Own GPU infrastructure, operations, often lower capability |

Questions to ask every provider:

- Where are inputs and outputs processed and stored? Is EU data residency contractually committed?
- Is data used for training? Business and API offerings usually exclude it.
- How long is data retained, for example for abuse monitoring? Is there a zero-retention option?
- Which sub-processors are involved?
- Which certifications and security attestations are available?

Processing in the EU lowers the risk considerably but does not settle every question: for providers headquartered in the US, possible access by US authorities remains a topic of debate. Open-weight models such as Llama, Mistral, Qwen or Gemma can run entirely in your own data centre or with a European host.

## EU AI Act: what applies to chatbots

The EU AI Act applies in stages. Two obligations matter for typical company assistants:

- **AI literacy (Article 4):** since 2 February 2025, providers and deployers of AI systems have had to look after the AI literacy of staff who work with those systems. The AI omnibus adopted in 2026 (Regulation (EU) 2026/1744) softened the duty: organisations must now take measures that support the development of AI literacy rather than ensure a specific level. Training and clear internal rules remain sensible.
- **Transparency (Article 50):** since 2 August 2026, AI systems that interact directly with people must be designed so that people know they are talking to an AI, unless that is obvious. Further labelling duties apply to AI-generated content.

A chatbot for product questions or internal knowledge is generally not a high-risk system. That can change when AI helps decide on job applications, credit or access to essential services. Have such use cases assessed separately.

## Checklist for the start

- Purpose and audience defined in writing
- Data sources selected, sensitive areas excluded
- Permissions enforced all the way into retrieval
- Legal basis documented, works council involved for internal tools where applicable
- Data processing agreements with all providers
- Location, retention and training exclusion clarified
- Retention periods for conversations and logs defined
- Need for a DPIA assessed
- "You are talking to an AI" notice in the chat window
- Privacy notice updated
- Answers cite sources; uncertain questions are handed over to people
- Staff trained on capabilities and limits

## Start small: a six-week pilot

A tightly scoped pilot quickly shows whether the effort pays off:

1. **Week 1:** pick one use case, such as questions about product documentation, and collect 50 typical questions with expected answers.
2. **Week 2:** settle data sources and provider, clarify contracts and data protection.
3. **Weeks 3–4:** build a RAG prototype and tune chunk size and retrieval against the test questions.
4. **Week 5:** test with a small user group and review wrong or missing answers.
5. **Week 6:** decide whether to expand, adjust or stop, based on the test results.

The test questions are your key asset: after every change they show whether the assistant improved.

## The takeaway

A good AI assistant is above all a good data project: clear sources, clean permissions, traceable answers and contracts that match the risk. Settle these basics in the pilot and you can expand with confidence. We support you from concept to operation as part of our [AI and automation services](https://degoya.digital/en/services/ai-automation). For choosing and connecting your content systems, see our [CMS consulting](https://degoya.digital/en/services/consulting-cms), or [contact us directly](https://degoya.digital/en/contact).

## Frequently asked questions

Will my data be used to train the AI model?

With the business and API offerings of major providers, this is usually excluded by default. Check the contract and privacy terms of the specific service anyway, because consumer apps often follow different rules.

Do I have to disclose that users are talking to an AI?

Yes. Under Article 50 of the EU AI Act, since 2 August 2026 people must generally be informed that they are interacting with an AI system unless that is obvious. A clear notice in the chat window is the simplest solution.

Is RAG or fine-tuning better for company knowledge?

For knowledge that changes or is subject to access rights, RAG is almost always the better choice. Documents can be updated or deleted without retraining a model, and answers can cite their sources.

## About the author: Alexander Herling

Founder and owner of DEGOYA.digital. A deep affinity for technology since early childhood and his first contact with 8-bit computers make him a digital native from day one. The command line is his friend. 35 years of experience in marketing and web development, specialising in SEO techniques.

Related service

[AI & automation tools](https://degoya.digital/en/services/ai-automation)

[Back to the blog](https://degoya.digital/en/blog)

Source: https://degoya.digital/en/blog/ai-assistants-own-data-privacy