A story of one who moved out to legally use an LLM in a 12-man SME and, if possible, also to certify according to ISO 41001. This is, so to speak, the project plan ideas collection and the entry-level structure brainstorming.
Note on the use of this document
This project plan was created as a structured working basis and classifies the technical, organizational and legal steps for the introduction of Open WebUI + Ollama as an on-premise AI solution as well as the way to a possible ISO/IEC 42001 certification.
REDDIT IANAL (I am not a Lawyer) Disclaimer: The legal assessments on GDPR, NIS2/BSIG-neu and the EU AI Act are based on the publicly available legal status of August 2026 and do not replace legal advice in individual cases. For legally binding statements (in particular on the appointment of a data protection officer, the need for a data protection impact assessment and the NIS2 applicability), it is recommended to consult a law firm specialising in data protection and IT security law or a suitably qualified external consultant.
contents
1. Executive Summary
2. Initial situation and objectives
3. Legal framework overview
4. Detailed analysis: Data protection (GDPR / BDSG)
5. Detailed analysis: NIS2 / BSIG-new
6. Detailed analysis: EU AI Act
7. ISO/IEC 42001 – Pathway to certification
8. Technical architecture and implementation
9. Project organization and roles
Phase plan and milestones
11. Risk management
12. Cost and resource planning
13. Success criteria
14. Next steps (first 30 days)
15. Annex 18
1. Executive Summary
The company plans to introduce Open WebUI as a self-hosted, browser-based interface for local AI language models (based on: Ollama) to provide the 12 employees with a data protection-compliant AI assistant for text work, knowledge search in technical documentation (RAG) and further use cases (completely on their own infrastructure) without transferring company or customer data to external cloud providers.
The present project plan has three parallel objectives:
- Technical introduction: robust, high-performance and securely operated on-premise AI stack (Open WebUI, Ollama, RAG knowledge database, access control).
- Legal compliance from the outset: Consideration of GDPR/BDSG, the EU AI Act and a well-founded, company-specific assessment of the NIS2 applicability (BSIG-new).
- Ability to certify: Development of an AI management system (AIMS) that meets the requirements of ISO/IEC 42001 and can be certified externally.
Key findings in advance:
- On-premise operation is the core advantage under data protection law: As long as no data flows to external model APIs, web search plugins or cloud services, no order processing arises in accordance with Art. 28 GDPR vis-à-vis an AI provider. This advantage must be technically and organisationally secured (see Chapters 4 and 8).
- As things stand today, NIS2 should not be directly mandatory: With 12 employees, the company is expected to be below the thresholds for ‘important’ or ‘particularly important entities’. Voluntary guidance on the NIS2 baseline measures is nevertheless recommended, inter alia due to possible supply chain requirements of larger customers (see Chapter 5).
- The EU AI Act already affects the company in parts: in particular, the obligation to impart AI skills (Article 4 of the AI Regulation), which has been applicable since February 2025, irrespective of the size of the company. High-risk obligations are not relevant for the intended use case (see Chapter 6).
- ISO/IEC 42001 is voluntary but strategic: As proof to customers from the measurement and manufacturing technology supply chain and as a structural basis for legally compliant AI governance (see Chapter 7).
The planned total period until certification maturity is approximately 10-12 months, divided into eight project phases (Chapter 10). A first rough estimate of costs can be found in Chapter 12.
2. Initial situation and objectives
2.1 Initial situation
The company is an SME with 12 employees in the field of industrial measurement technology. Typical data categories in such an establishment include technical drawings and design data, measurement logs and calibration data, customer documentation (partly under NDA), internal process and quality documentation (e.g. in the context of ISO 9001), and personal data of employees, customers and suppliers. These data are regularly classified as business and trade secrets within the meaning of the Trade Secrets Act (GeschGehG) and are often subject to additional contractual confidentiality obligations vis-à-vis customers from the manufacturing industry.
Cloud-based AI chat tools (e.g. ChatGPT, Copilot, Gemini in the standard configuration) would mean a transfer to non-European or external providers in this data situation, which is risky both under data protection law and contractually in a sensitive B2B environment with confidentiality obligations. A self-hosted, on-premise-operated solution such as Open WebUI in combination with Ollama basically avoids this data outflow, as the model and data remain completely on their own infrastructure.
2.2 Project objectives
- Provision of an internal, GDPR-compliant AI wizard for all 12 employees (text drafts, summaries, research in internal documentation, perspective support for offer and report texts).
- Development of an RAG-supported knowledge base based on internal technical documentation, manuals and standards, without this content leaving the company.
- Establishment of a legally compliant operational basis (data protection, AI usage policy, role and rights concept).
- Establishing an AI management system (AIMS) and obtaining certification according to ISO/IEC 42001 within about 12 months after the start of the project.
- Creation of a reference point for customer inquiries and tenders, which increasingly require proof of responsible use of AI and information security.
2.3 Delimitation (non-objectives)
- No use of AI for automated decision-making vis-à-vis customers or applicants (no profiling, no automated individual decisions within the meaning of Article 22 GDPR).
- No embedding of the AI component in physical measurement products or production control as part of this project (this would necessitate reclassification as a high-risk AI system under Annex I to the EU AI Regulation, see Chapter 6.3).
- No productive use of cloud-based additional AI functions (external web search, external image generation, external language APIs) in the basic configuration, as these would be contrary to the on-premise approach (see Chapter 8.6).
- No personnel decisions or performance evaluations based on AI evaluations of employee data.
3. Overview of the legal framework
The following table summarizes which regulations are relevant for the project, whether they are binding for a company of this size and what specifically follows for the project. The detailed analysis is carried out in chapters 4 to 7.
| Rules and regulations | binding | Core obligation | Related to the project |
| GDPR / BDSG | Binding (always) | Legal, secure processing of personal data | Processing directory, TOMs, DSFA if applicable, Data Protection Officer if applicable |
| NIS2 / BSIG-new | Probably not directly obligatory | Cybersecurity risk management, reporting obligations | Voluntary orientation recommended; Check supply chain relevance to larger customers |
| EU AI Act | Partially binding | AI competence (Art. 4), prohibitions (Art. 5), transparency (Art. 50) | the obligation to provide training in accordance with Article 4; High-risk obligations not relevant |
| ISO/IEC 42001 | Voluntary | Development of an AI management system (AIMS) | Certification goal of the company is structured the entire project |
| GeschGehG | Mandatory | Appropriate confidentiality measures | Access and logging concept for design/measurement data |
| BetrVG (if works council exists) | Conditionally binding | Co-determination in systems with monitoring potential (Section 87(1)(6)) | Early involvement, AI usage policy, if applicable, as an operating agreement |
Note: This overview does not replace a legal case-by-case assessment. In particular, the NIS2 classification depends not only on the number of employees but also on the company's annual turnover and balance sheet total, which are not yet known here (see Chapter 5.2).
4. Detailed analysis: Data protection (GDPR / BDSG)
4.1 Responsibility and legal bases
As the operator of the on-premise system, the company is ‘controller’ under data protection law within the meaning of Article 4(7) GDPR for all personal data processed via Open WebUI. The following may be considered as legal bases:
- Art. 6(1)(f) GDPR (legitimate interest) for general use as an efficiency and research tool, provided that a balance of interests is documented.
- § 26 BDSG in conjunction with Art. 88 GDPR for the processing of employee data in the context of use (e.g. usage logs, prompt history of individual employees).
- Art. 6(1)(b) GDPR , insofar as customer data is processed in the context of the execution of the contract (e.g. preparation of test reports).
Recommendation: The specific legal basis should be documented in the processing directory for each use case (see 4.5).
4.2 Order processing is the central advantage of on-premise operation
As long as Open WebUI and the connected language models are operated exclusively locally on their own infrastructure and no requests are sent to external model APIs (e.g. OpenAI, Anthropic, Google), no order processing arises under Article 28 GDPR vis-à-vis an AI provider. In principle, therefore, no order processing contract (DPA) and no examination of a third-country transfer in accordance with Chapter V GDPR for core AI operations must be carried out.
Note: This advantage only applies as long as the configuration is consistently adhered to. As soon as optional cloud functions are activated in Open WebUI (external web search providers, external image/language generation or an additional cloud model as fallback), data is once again transmitted to third parties for these functions, which requires an AVV and, if necessary, a transfer impact assessment (see Chapter 8.6). This should be technically (deactivation in the admin configuration) and organizationally (AI usage policy).
4.3 Data protection impact assessment (DSFA, Art. 35 GDPR)
Whether a DSFA is mandatory depends on the specific risk of processing. There are several indications for the use of an AI application for a DSFA obligation or for a recommended voluntary implementation:
- Use of new technologies (Article 35(1) GDPR) AI language models are regularly considered as such according to widespread supervisory practice.
- Possible systematic and comprehensive assessment of personal aspects, provided that the AI would also be applied to personnel documents or applications in the future (excluded in the current scope, see 2.3).
- In their guidance on AI applications, German supervisory authorities (Data Protection Conference, DSK) regularly recommend the implementation of a DSFA, even below the mandatory legal threshold, as proof of accountability (Article 5(2) GDPR).
Recommendation: A DSFA should be carried out as part of project phase 1, regardless of the final legal classification of its obligation. It also provides an essential building block for the subsequent ISO/IEC 42001 certification (risk assessment, see Chapter 7.5) and for any NIS2 risk management (Chapter 5).
4.4 Obligation to appoint a data protection officer (§ 38 BDSG)
Under the first sentence of Paragraph 38(1) of the BDSG, an order obligation generally only exists if at least 20 persons are permanently involved in the automated processing of personal data. With 12 employees, this threshold should generally not be reached.
Important exception: Pursuant to the second sentence of Paragraph 38(1) of the BDSG, the obligation to place an order applies irrespective of the number of employees as soon as processing is carried out which is subject to a data protection impact assessment pursuant to Article 35 of the GDPR. If a DSFA obligation is affirmed for the AI project as recommended in 4.3, this may also result in an obligation to appoint a data protection officer regardless of the size of the company.
- This question should be clarified at an early stage during the DSFA pre-examination with a specialist.
- Regardless of a legal obligation, the voluntary designation of a responsible contact person (internal or external data protection officer) is recommended for an SME of this size in the AI context and is recommended by ISO/IEC 42001 in the sense of clear roles and responsibilities anyway (see Chapter 9).
Note: At federal level, it is currently being discussed that § 38(1) BDSG should be deleted without replacement and that the obligation to place orders should be based exclusively on the risk-based approach of Article 37 GDPR (announcement in the context of the ‘Federal Modernisation Agenda’ of 4 April 2018). December 2025, implementation announced by the end of 2026, as of August 2026 not yet in force). The current legal situation should be re-examined at the time of implementation.
4.5 List of processing activities (Art. 30 GDPR)
For each AI-based use case (e.g. ‘internal chat assistant’, ‘RAG knowledge search technical documentation’, ‘customer communication text drafts’), a separate entry must be created in the processing directory, with information on the purpose, groups of persons concerned, data categories, legal basis, storage period, technical and organisational measures and, where applicable, recipients.
4.6 Technical and organizational measures (Art. 32 GDPR)
In particular, the following measures must be implemented and documented for on-premise operation:
- Encryption of data at rest (database, vector storage, file system) and in transit (TLS/SSL via the reverse proxy, see chapter 8.5).
- Role and rights concept in Open WebUI (admin vs. user roles, access restriction to knowledge bases as required).
- Logging of security-relevant events (accesses, administrative changes) taking into account the purpose limitation. Furthermore, no behavioural or performance control of individual employees.
- Regular, encrypted backups with a defined recovery concept.
- Network segmentation: The AI server is to be isolated in the internal network, no direct access from the Internet without VPN or secured reverse proxy.
- Patch and version management for Open WebUI, Ollama and the operating system base.
4.7 Rights of data subjects and deletion concept
Since RAG knowledge databases can also contain personal data (e.g. contact persons in customer documents, names in e-mail attachments), a deletion concept is required that specifies how requests for information, correction and deletion (Articles 15-17 GDPR) can be technically implemented in the vector database. Recommendation: provide for the screening/anonymisation of obviously personal but not required content before introducing documents into the knowledge base.
4.8 Employee data protection and codetermination
The introduction of an AI system, which can in principle be logged and evaluated, may be subject to co-determination under § 87(1)(6) BetrVG, provided that there is a works council in the company (not necessarily available for 12 employees, but possible). If a works council exists, it should be involved at an early stage and an operating agreement on the use of AI should be concluded. If there is no works council, it is still recommended to create a written AI usage policy that transparently regulates what is logged and why (see Chapter 4.9).
4.9 AI Usage Policy for Employees
An AI Utilisation Directive should be established as a binding internal document, covering, inter alia:
- Permissible and inadmissible use cases (e.g. no entry of strategic trade secrets of third parties without release, even if they remain on-premise).
- Prohibition of circumvention of technical protection measures (e.g. private use of external AI tools for corporate content subject to confidentiality).
- Handling of AI-generated content (labelling obligations, obligation to carry out technical checks before re-use, in particular for measurement and test data).
- Transparency information on logging and evaluation according to Art. 13/14 GDPR.
5. Detailed analysis: NIS2 / BSIG-new
5.1 Legal background
The EU Directive (EU) 2022/2555 (NIS2) was transposed in Germany with a significant delay by the ‘Law on the implementation of the NIS 2 Directive and on the regulation of essential principles of information security management in the federal administration’ (NIS2UmsuCG). The law was passed on the 5th. It was promulgated in the Federal Law Gazette on December 6, 2025. entered into force on 1 December 2025 without a transitional period; it fundamentally amends the BSI Act (BSIG-new). Across Germany, it is estimated that around 29,500 companies are directly affected; the obligation to register with the BSI Portal (activated on 6 January 2026) expired on 6 March 2026 for affected entities.
5.2 Applicability test for the company
The NIS2 obligations apply to ‘important’ and ‘particularly important entities’. The classification is carried out in two stages: first on sectoral affiliation (Annex I/II to the Directive), then on size classes.
Step 1 – Sector affiliation
The manufacture of measuring, control, navigation and similar instruments falls under NACE Group C26 (‘Manufacture of data processing equipment, electronic and optical products’), which is explicitly listed in Annex II to the NIS2 Directive as a sub-sector of the ‘manufacturing industry’ as another critical sector (‘important entity’). An industrial metrology company is therefore generally within the scope of NIS2 in terms of sectoral scope, but the final applicability depends on the size class (step 2).
Step 2 – Size class
NIS2 generally excludes micro and small enterprises from its scope. The EU SME definition is relevant:
| category | Employees | Annual turnover | or balance sheet total | NIS2 status |
| micro | < 10 | ≤ €2 million | ≤ €2 million | Regularly excepted |
| Small business | < 50 | ≤ €10 million | ≤ €10 million | Regularly excepted |
| Medium-sized enterprise | < 250 | ≤ €50 million | ≤ €43 million | ‘Important facility’ possible |
Classification: With 12 employees, the company is below the 50-person threshold for ‘small companies’. If, in addition, the annual turnover or balance sheet total does not exceed €10 million (which is likely to be the rule for this size of business, but must be confirmed on a company-specific basis), the direct NIS2 obligations are unlikely to apply according to current knowledge.
Irrespective of the size exemption, NIS2 nevertheless applies to certain types of entities, regardless of their size (including operators of public telecommunications networks, qualified trust service providers, DNS service providers/TLD registrars, public administrations, entities classified as critical under the KRITIS Roofing Act and entities that are the sole providers of a service in Germany that is important for the maintenance of critical societal or economic activities). None of these exceptions should typically apply to a measurement SME of this size; However, a short assessment in project phase 1 is recommended (see Annex A).
5.3 Indirect relevance: Supply chain security
Even without its own NIS2 obligation, the company may be indirectly affected: NIS2 obliged customers (e.g. major mechanical engineering or automotive suppliers who are themselves considered as ‘important’ or ‘particularly important entities’) need to assess the security of their supply chain (Article 21 of the NIS2 Directive) and are increasingly contractually passing on safety requirements to suppliers, including smaller partners such as a measurement SME. Documented information security and AI risk management is increasingly becoming a competitive and tendering factor here, regardless of one's own NIS2 obligation.
5.4 Recommendation for the project
- Do not make a formal NIS2 registration as long as the thresholds are not met (unnecessary bureaucratic effort).
- Use the NIS2 basic measures according to § 30 BSIG-neu (including risk management, incident detection and reporting processes, backup and emergency management, cryptography and encryption concepts, access control, training, supply chain security) voluntarily as orientation for the technical protection of the AI system. These are largely in line with the controls required anyway for ISO/IEC 42001 and Article 32 GDPR (see Chapter 4.6, 7.5, 8.5).
- The possibility of voluntary registration with the BSI (for non-obliged companies under the new BSIG) can optionally be checked if the company wants to actively advertise with increased cyber resilience to customers.
- The size classification should be reassessed in the event of future growth (exceeding 50 employees or € 10 million in turnover/balance sheet total), as this may result in a NIS2 obligation.
6. Detailed analysis: EU AI Act
6.1 Role of the Company
In the planned use of Open WebUI with pre-trained open language models (e.g. via Ollama), the company acts under data protection and product law as an ‘operator’ (deployer) within the meaning of Article 3(4) of the AI Regulation, not as a ‘provider’ (provider). Provider obligations (conformity assessment, technical documentation of the model) apply to model manufacturers, not to the applying SME, unless the company would adapt its own models so fundamentally or pass them on under its own name that it would become the supplier itself (not provided for in the planned scope).
6.2 Prohibited practices (Art. 5 AI Regulation)
The bans on certain AI practices (including social scoring, certain forms of biometric categorisation, manipulative systems) have been applicable since 2 February 2025. For the planned internal assistance and knowledge search use case, no contact with these prohibitions is apparent.
6.3 Classification as a high-risk AI system (Annex III)
An internal AI assistant for text work and technical knowledge search does not fall under the high-risk use cases listed in Annex III of the AI Regulation (including staff selection, credit assessment, biometric identification, critical infrastructure management). As long as the demarcation described in Chapter 2.3 is complied with, the use case is not high-risk.
Note: Irrespective of this, the so-called ‘Digital Omnibus on AI’ – a political agreement reached by the Council and the European Parliament on 7 May 2026 – has the full obligations for stand-alone high-risk AI systems set out in Annex III anyway from August 2026 to the 2nd. December 2027 postponed (Annex I products: 2 August 2028). The formal adoption of this document in the Official Journal of the EU was pending (August 2026); Until the final publication, the original schedule will continue to apply formally. For the use case described here, this is secondary anyway, as there is no high-risk classification.
6.4 Duties that already apply
Article 4 of the AI Regulation – AI literacy: Since 2 February 2025, providers and operators of AI systems have been required to ensure sufficient AI competence in their staff and other persons involved in the operation and use of AI systems on behalf of them. This obligation applies regardless of the size of the company and is enshrined in the project plan as a mandatory task (training all users before starting production) – see Chapter 10, Phase 4.
Article 53 of the AI Regulation – GPAI obligations: Transparency and documentation requirements for General Purpose AI models have been applicable since August 2025 and enforcement will start in August 2026. These obligations are primarily imposed on model providers (e.g. developers of the open models used), not on the user SME as operator.
Article 50 of the AI Regulation – Transparency obligations: From 2 August 2026, the obligation to identify AI-generated or modified content applies, provided that it is used externally (e.g. vis-à-vis customers). This is relevant once the AI Assistant is used to prepare customer-facing texts and should be regulated in the AI Usage Directive (Chapter 4.9).
6.5 Conclusion
The direct legal obligations under the EU AI Act are manageable for the intended use case. The most important concrete task is to ensure AI competence in accordance with Article 4 of the AI Regulation by means of training. Governance elements that make sense for the AI Act anyway (roles, risk assessment, documentation, transparency) are largely in line with the requirements of ISO/IEC 42001 (Chapter 7) and should be built up synergistically.
7. ISO/IEC 42001 – Pathway to certification
7.1 What is ISO/IEC 42001?
ISO/IEC 42001:2023 is the first international standard for an Artificial Intelligence Management System (AIMS). It follows ISO 9001 (quality) and ISO/IEC 27001 (information security) of the high-level structure (HLS) of the ISO management system standards and is structured according to the PDCA cycle (Plan-Do-Check-Act). The standard is applicable across all industries and is designed for organizations of all sizes that develop, operate or use AI systems.
7.2 Benefits to the Company
- Structured evidence of a responsible, controlled use of AI towards customers, in particular towards larger supplier partners and contracting authorities, where ISO 42001 is increasingly in demand as a tender criterion or trust signal.
- Systematic preparation for future EU AI Act obligations: AIMS documentation (risk assessment, roles, controls) can be re-used for proof of conformity.
- Synergies with existing management systems: Measurement technology companies often already have a quality management system according to ISO 9001 or/and ISO/IEC 17025 (calibration laboratories), the HLS structure allows for extensive integration instead of a parallel system.
- Internal impact: Clear responsibilities and documented processes reduce the risk of misuse, data breaches and loss of knowledge in an SME of this size.
7.3 Define scope (scope)
Recommendation for the scope of certification: ‘Provision and operation of an internal, on-premise hosted AI assistance system based on Open WebUI and locally operated language models, including retrieval-based knowledge search (RAG) for the company’s employees on site [location].’ A narrow, realistic scope significantly accelerates certification over an enterprise-wide AIMS for all conceivable AI applications.
7.4 Certification Roadmap
For an SME of this size, a realistic timeframe of approximately 9-12 months to certification maturity should be considered (references indicate 6-18 months depending on the starting point). The roadmap is divided into five key steps enshrined in the phase plan (Chapter 10):
| step | contents | result |
| 1. Gap analysis | Matching actual state against ISO/IEC 42001 Chapters 4-10 and Annex A | List of measures with prioritization |
| 2. Project team & Governance | Appointment of AIMS controller, adoption of AI policy | AI Policy, Role Matrix |
| 3. Risk assessment | AI-specific risk and impact assessment per use case | Risk register (Chapter 11) |
| 4. documentation | AIMS manual, procedural instructions, evidence (training, audits) | Full AIMS documentation |
| 5. Internal Audits & Review | Internal audit, management review, corrective actions | Audit report, approval for certification |
7.5 Key requirements at a glance
As with ISO 9001/27001, the standard is divided into seven main chapters:
- Context of the organization (chapter 4): Interested parties, scope definition.
- Leadership (chapter 5): AI policy, roles and responsibilities – often in dual roles for 12 employees (see Chapter 9).
- Planning (chapter 6): Risks and opportunities, AI objectives, AI impact assessment per system.
- Support (chapter 7): Resources, competence (interface to Art. 4 AI Regulation), awareness, communication, documented information.
- Operation (chapter 8): operational planning and management of the AI lifecycle, supplier/third-party management.
- Assessment of performance (chapter 9): Monitoring, internal audit, management review.
- Improvement (chapter 10): Dealing with non-conformities, continuous improvement.
In addition, Annex A defines specific controls (e.g. on AI Directive, resource management, data for AI systems, transparency towards stakeholders, use of third-party/open source components such as Ollama models, and assessment of societal impacts). These controls should be mapped 1:1 to the technical architecture described in Chapter 8.
7.6 Certification process
- Stage 1 audit: Document review by the certification body (audit of AIMS manual, AI policy, risk register, procedural instructions).
- Stage 2 audit: On-the-spot or remote effectiveness testing (interviews, sampling, technical verification of the controls implemented).
- Issuance of certificates upon successful audit, validity of normally 3 years.
- Annual surveillance audits to maintain the certificate.
- Re-certification audit at the end of the three-year cycle.
7.7 Choice of certification body
It is recommended that only a certification body accredited by the German Accreditation Body (DAkkS) or an equivalent European accreditation body (e.g. TÜV companies, DNV, DEKRA or comparable providers) be commissioned. The concrete selection should be made through at least two to three comparative offers, including experience with SMEs and manufacturing companies).
8. Technical architecture and implementation
The technical basis follows the approach proposed by the company (Open WebUI + Ollama, Docker-based) and is supplemented by the necessary safeguards for a secure, certified SME setup.
8.1 Target architecture
We recommend a Docker Compose deployment on a dedicated server physically in the enterprise (alternatively: Mini Server/Mac Mini Class for entry, with option to upgrade). The LLM backend does not require an outgoing Internet connection for core operation. A largely netzisolierte ("air-gap-close") operation is possible and also recommended from a data protection and security point of view.
8.2 Component overview
| component | function | Recommendation for the project |
| Open WebUI | Frontend, user/rights management, RAG interface | Run version-pinned (no ‘:main’ tag), regular controlled updates |
| Ollama | Local model runtime | Model selection according to Chapter 8.4, no automatic cloud connection |
| Vector Database (RAG) | Storage/search in knowledge base | Start with integrated ChromaDB; Consider Qdrant as Document Stock Grows |
| Reverse Proxy (nginx/Caddy) | Encrypted, controlled access in the company network | TLS certificate, access only from internal network/VPN |
| User management | authentication | If possible, connection to existing directory (e.g. via SSO/LDAP), otherwise individual accounts per employee |
8.3 Hardware planning
For 12 users with predominantly sequential use (no high-load mass operation), a server with a consumer to prosumer GPU (e.g. class RTX 4000/4060 Ti 16 GB or comparable) is usually sufficient for models in the range of 7-14 billion parameters in quantified form. For higher quality requirements or more simultaneous use, a GPU with more VRAM (24 GB+) should be budgeted. Pure CPU operation is possible, but noticeably slower for productive use with several simultaneous users and is not recommended. The concrete dimensioning should be validated after a short pilot phase (phase 3) on the basis of the actual load on use.
8.4 Model selection - Criteria
The concrete model selection is a technical decision in the course of the project, but should be made and documented on the basis of the following criteria (also relevant for ISO 42001 Annex A, ‘Data for AI systems’):
- Open, clearly licensed model weights (check license terms for commercial use).
- Quality in German and in technical-technical contexts (measuring technology terminology).
- Resource requirements to match the available hardware (see 8.3).
- Traceable origin and update/support history of the model provider (documentation obligation under Art. 53 AI Regulation applies to the provider, but should serve as a selection criterion for the operator).
- No automatic telemetry or contacting external servers in regular operation.
8.5 Safety concept
- Network segmentation (own VLAN for the AI server), no direct Internet access to the backend.
- Access only via secured reverse proxy with TLS certificate, ideally only accessible from the internal network or via company VPN.
- Role/rights concept: Admin role limited to IT managers, standard users without access to system configuration.
- Logging of security-relevant events with a defined, data protection-compliant retention period.
- Regular, audited backups (3-2-1 principle recommended) and documented recovery process.
- Defined patch and update process for all components (Open WebUI, Ollama, operating system, container base images).
- These measures are broadly in line with the voluntary NIS2 baseline measures (Chapter 5.4) and Article 32 GDPR requirements (Chapter 4.6).
8.6 Control and Disabling Critical Cloud Features
Open WebUI offers numerous optional extensions (external web search providers, external image generation, external voice/transcription services, cloud models as an additional option). These functions are efficient, but contradict the on-premise and data protection concept of this project as soon as they transmit personal or confidential data to third parties.
Note: Recommendation: Disable all external/cloud-based extensions in the administration interface by default. If individual cloud functions are nevertheless to be used in the future (e.g. an external web search for non-confidential searches), this must be assessed in advance under data protection law (contract processing contract, transfer impact assessment if applicable in relation to third countries) and explicitly regulated in the AI Usage Directive.
9. Project organization and roles
In a company with 12 employees, a complete separation of all roles between different people is unrealistic. However, ISO/IEC 42001 requires clearly documented responsibilities, even if a person performs multiple roles in personal union. The decisive factor is the written fixation, not the number of heads.
| role | responsibility | Appointment recommendation (12-MA-SME) |
| Project management / management | Overall responsibility, release AI policy, budget | management |
| AIMS/AI Manager(s) | Setup and maintenance of the AI management system, contact person for audits | Management or appointed manager, if necessary with external support |
| IT Administration | Technical operation, security configuration, patch management | Internal IT responsibility or external IT service provider |
| Data protection contact person / DSB | DSFA, processing directory, data subject enquiries | Externally appointed Data Protection Officer recommended (see Chapter 4.4) |
| Key users / multipliers | Technical feedback from the departments, training support | 1–2 experienced practitioners |
| External consulting ISO 42001 (optional) | Gap analysis, audit preparation | Participate on a case-by-case basis, especially before Stage 1 audit |
Phase plan and milestones
The total period up to certification maturity is approximately 10-12 months. Several phases are deliberately running in parallel in order to realistically represent the limited personnel capacities of a 12-person operation. The development of the AIMS (Phase 5) starts already during the technical implementation and runs up to the internal audit maturity.
| # | Phase | month | Main results |
| 0 | Project initiation & scoping | 1 | Project order, goal definition, scope definition ISO 42001, budget approval |
| 1 | Basic legal work | 1–2 | DSFA, clarification of DSB obligation, processing directory, NIS2 short check, AI usage policy (draft) |
| 2 | Technical concept & Procurement | 2–3 | Hardware selection, network/security concept, model selection, procurement |
| 3 | Implementation & Pilot operation | 3-4 | Installation Open WebUI/Ollama, RAG setup, test operation with core group |
| 4 | Rollout & Training | 4-5 | Training of all employees (incl. Art. 4 AI Regulation), productive rollout, AI Usage Directive final |
| 5 | Structure AIMS according to ISO 42001 | 3-8 | AI policy, role matrix, risk register, documentation (runs in parallel from pilot start) |
| 6 | Internal audits & management review | 8-9 | Internal audit, corrective actions, approval by management |
| 7 | Certification audit (Stage 1 + 2) | 9-11 | Selection of certification body, document review, on-site/remote audit |
| 8 | Certificate issuance & Operation | from 11 to 12 | Certificate, ongoing operation, annual monitoring audit, continuous improvement |
11. Risk management
The following risk register also forms a component of the risk assessment required by ISO/IEC 42001 Chapter 6 and should be continuously updated in the course of the project.
| risk | category | It's true. | impact | measure |
| Accidental activation of cloud plugins (web search, external API) | privacy | appropriation | High | Block admin configuration, AI usage policy, regular configuration check |
| Low acceptance by employees | Organisation | appropriation | appropriation | Early involvement, comprehensible training, key users as multipliers |
| Insufficient hardware performance in practice | technique | appropriation | appropriation | Pilot phase for load validation before full expansion, planning hardware scalable |
| Know-how/secret outflow via prompt inputs in future cloud expansion | Data protection / GeschGehG | Low (when complying with 8.6) | High | Technical deactivation, clear policy, training on confidentiality obligations |
| Incorrect AI outputs in technical interpretation of measurement data (hallucination) | Technical / quality | appropriation | High | Mandatory obligation for technical testing before re-use, no automated release of AI texts in test reports |
| Delay due to limited internal capacity (12-MA operation) | project | High | appropriation | Realistic scheduling, ad hoc external support for data protection and ISO 42001 |
| Model deprecation / discontinuation of the license of a used open model | technique | low | appropriation | Document model selection, provide for change to alternative model as emergency plan |
| Failure to reach certification maturity in the planned period | Project / certification | appropriation | appropriation | Early gap analysis, iterative internal audits instead of big bang before stage 1 |
12. Cost and resource planning
The following information is a rough guideline based on market-standard bandwidths for consulting, hardware and certification in Germany (as of 2026) and does not replace specific offers. It is recommended to obtain at least two to three comparative offers for the cost-intensive positions (consultation, certification) before budget approval.
| position | type | Rough estimate | remark |
| Hardware (server/GPU, network components) | Unique | approx. 3,000 – 12,000 € | depending on model size and user load; Software itself is open source (free of charge) |
| External IT setup/setup support (optional) | Unique | approx. 1,500 – 5,000 € | Docker setup, backup, RAG configuration, if not covered internally |
| External Data Protection Officer (if not internal) | Ongoing, annual | approx. 2,000 – 6,000 €/year | depending on size and provider, for an SME of this size |
| External consulting ISO 42001 (Gap analysis, audit preparation) | Unique | approx. 4,000 – 15,000 € | strongly dependent on internal prior knowledge and existing management systems (e.g. ISO 9001) |
| Certification audit (Stage 1 + 2) | One-time, then annually (monitoring) | approx. 4,000 – 10,000 € | depending on the certification body and scope; Annual monitoring audits are cheaper |
| Training Employees (AI competence, use) | Unique + ongoing | approx. 1.000 – 3.000 € | It can be carried out partly internally. |
| Internal personnel time (project management, IT, departments) | Continuously over the duration of the project | Not quantified in euro | Significant factor for 12 employees – realistic capacity planning required |
13. Success criteria
- All 12 employees are trained and actively use the AI assistant for at least one documented use case.
- No reported data protection or secrecy incidents related to AI use.
- Processing directory, DSFA and AI usage policy are fully documented and up-to-date.
- Successful completion of Stage 1 and Stage 2 audits and issuance of the ISO/IEC 42001 certificate within the planned timeframe.
- Positive employee feedback on usability (e.g. via short internal survey after 1 and 3 months of productive operation).
- Demonstrable time savings in at least one defined use case (e.g. document research, draft texts, standard support cases).
14. Next steps (first 30 days)
- Formally approve the project order by the management, roughly define the budget.
- Appoint AIMS/AI managers (also possible in personal union with management).
- Contact legal advice or external data protection consultants for DSFA preliminary examination and clarification of the DSB obligation.
- Perform and document NIS2 short check (Annex A) internally, in particular check annual turnover/balance sheet total against the thresholds.
- Capture initial technical requirements (number of users, desired use cases, existing hardware) and obtain offers for server hardware.
- Start a rough market survey of ISO/IEC 42001 certification bodies and, if necessary, consulting companies (offer comparison).
- Kick-off meeting with all employees for project announcement and clarification of expectations.
15. Appendix
Annex A – NIS2 applicability check
- Does the company usually employ 50 or more people? (If not, keep checking)
- Does the annual turnover exceed €10 million or the balance sheet total exceed €10 million? (If no → NIS2 is generally not applicable)
- Is the company the sole provider of a critical service for Germany, qualified trust service provider, TLD/DNS provider, telecommunications provider or classified as critical under the KRITIS Roofing Act? (If no → no size-independent special obligation)
- Does a significant customer contractually require NIS2-related proof of security in the supply chain? (If yes → Voluntary guidance on NIS2 baseline measures recommended, regardless of own obligation)
Annex B – DSFA pre-examination (short check under Article 35(3) GDPR / DSK criteria)
- Are behavioral or performance data of individual employees systematically evaluated?
- Are special categories of personal data (Art. 9 GDPR) processed or stored in the knowledge base?
- Is a novel technology with an unclear risk assessment used (AI language models are regularly considered as such)?
- Are data processed or searchable on a large scale (many documents/persons) automatically?
- If several ‘yes’ responses result in an increased risk, a DSFA must be carried out or at least documented why none is necessary.
Annex C – Main legal bases (overview)
- Regulation (EU) 2016/679 (General Data Protection Regulation, GDPR)
- Federal Data Protection Act (BDSG), in particular §§ 26, 38
- Directive (EU) 2022/2555 (NIS2 Directive) and the German transposing act NIS2UmsuCG (new BSI Act, BSIG-neu), in force since 6. December 2025
- Regulation (EU) 2024/1689 (AI Regulation / EU AI Act), in particular Articles 4, 5, 50, 53
- Digital Omnibus on AI → political agreement of 7 May 2026 on the postponement of the deadline for high-risk obligations (pending formal adoption as of August 2026)
- ISO/IEC 42001:2023 → Artificial intelligence Management system
- Law for the Protection of Trade Secrets (GeschGehG)
- Works Constitution Act (BetrVG), in particular § 87(1)(6) (if works council exists)