AI Enterprise LLM Platform Engineering
16 hours online course on designing and running an enterprise LLM platform: gateways, access control, data governance, LLM security and cost.
- Duration
- 4 sessions · 16 hours
- Language
- English
- Format
- Live online, with your own lab environment
- Class size
- Max 16
- Prerequisites
Hands-on experience running production services and a basic understanding of LLMs are expected. Experience hosting models is helpful but not required.
Course overview
Once AI moves from individual experiments to company-wide use, someone has to run it: who may use which model, where the data goes, what it costs and what happens when a provider fails. This course covers the design and operation of an enterprise platform that gives controlled, secure and observable access to hosted and self-operated language models.
We compare external model services, private environments and self-hosted inference, and cover model serving fundamentals such as quantization, accelerators, throughput and latency. You will work through gateway and control-plane patterns with routing, quotas and policy enforcement, identity and access management, data residency and audit requirements, and the security risks specific to LLMs and agents. We finish with observability, cost tracking and the lifecycle work of updates, rollbacks and incident response, so you can design a reference architecture for your own organization.
Agenda
-
Enterprise LLM architecture - Compare external model services, private environments and self-hosted inference, and define the responsibilities of an enterprise AI platform.
-
Model serving fundamentals - Model formats, inference, quantization, accelerators, memory, throughput, latency, scaling and the operational trade-offs behind model hosting.
-
Gateway and control-plane patterns - Provide unified access, provider abstraction, model routing, fallback, rate limits, quotas, budgets and policy enforcement through a central layer.
-
Identity and access management - Integrate user and workload identities, single sign-on, service identities, RBAC/ABAC and tenant-aware authorization.
-
Data protection and governance - Control data residency, retention, logging, personally identifiable information, confidential prompts and audit requirements.
-
LLM and agent security - Address prompt injection, data exfiltration, unsafe tool use, permission escalation and guardrails for model- and agent-based applications.
-
Observability, quality and cost - Measure traces, latency, token usage, errors, model behavior, quality signals and cost to support operations and chargeback/showback.•
-
Resilience and lifecycle operations - Plan model updates, compatibility testing, rollbacks, provider failures, capacity constraints and incident response for production AI services.
Who this course is for
This course is for platform, DevOps, security and architecture professionals who design, secure or operate AI services in their organization.
Outcome
Participants can design a reference architecture for an enterprise LLM platform, including access control, model routing, security controls, observability and operational lifecycle requirements.
What's included
- Digital documentation.
- Access to a virtual training center.
- Certificate of completion from croit.
Materials provided
- Digital documentation for every session.
- A virtual training centre with your own lab environment for the duration of the course.
Technical requirements
- A stable, high-speed internet connection: at least 25 Mbit/s downstream and 5 Mbit/s upstream.
- A computer that runs a modern web browser, for access to the lab.
- A headset, or a microphone with speakers, to take part in discussions and Q&A.
- A webcam is recommended; it makes the session more personal and interactive.
How booking works
- 1 Register, and tell us whether you need a purchase order.
- 2 Your invoice arrives within one working day and appears in your croit portal, where you can pay it by card or bank transfer.
- 3 Places are allocated in order of payment. If a run fills first, we offer you the next available date.
- 4 Joining details, the final schedule and your lab access arrive about a week before the start.
Questions before you book
Classes are capped at 16 participants so every question gets answered and the lab stays hands-on. A date runs once it has 3 paid bookings, and is then marked as guaranteed to run. If a date does not reach its minimum, it is rescheduled to the next available date and every booking moves with it.
There is no checkout on the website. Registering places an order; your invoice arrives within one working day and appears in your croit portal, where you can pay by card or bank transfer. Places are allocated in order of payment, so paying promptly secures your date. If a run fills first, we offer you the next available date.
Yes. Say so when you register and you receive a quote instead of an invoice, ready for procurement. Send us the purchase order number once you have it and we invoice against it. A purchase order usually takes three to six weeks to clear, so for a run starting sooner than that we may suggest a later date.
All prices are quoted excluding VAT. Businesses in Germany are invoiced with 19% German VAT. Businesses elsewhere in the EU with a valid VAT ID are invoiced net under the reverse-charge rule, and businesses outside the EU are invoiced without German VAT. Enter your VAT ID when you register so the invoice is right the first time.
A computer with a modern web browser and a stable connection, at least 25 Mbit/s downstream and 5 Mbit/s upstream, plus a headset or a microphone with speakers. A webcam is recommended. The lab runs in our virtual training centre, so nothing is installed on your machine.
We add dates through the year, and run private group trainings for teams. Tell us what you need and we will find a solution.
Private group trainingOther trainings
Other courses in this area, and the ones people take alongside it.