Technology
Everything above the metal.Built and run by Swarmio.
Swarmio Sovereign AI turns GPU capacity, Swarmio’s or your own, into governed, metered AI services: GPU as a Service, one governed gateway to every model, and applications for every team, with every unit of use attributed to the customer that consumed it.
The stack
Four layers.One control plane.
Applications, AI operations and the control plane are built, run and supported by Swarmio. Beneath them sit servers, GPUs, storage and network from any vendor, provided by Swarmio or by your organization. Swarmio Core, the control plane, has run in production since 2017.
Applications
Built, run and supported by SwarmioA starter catalogue for enterprise staff, plus any third-party, partner or in-house application, curated and hardened by Swarmio. Each is a private, isolated instance for one customer, wired into identity, secrets, network, models and metering at launch.
- Swarmio Chat
- Kleio
- Strata
- Koina
- Swarmio AI Coworkers
- Bring your own application
Conductor and Meter
Built, run and supported by SwarmioSwarmio Conductor, one governed gateway to every model, and Swarmio Meter, which attributes every token and every GPU-hour to the customer that consumed it.
- Swarmio Conductor
- Swarmio Meter
Swarmio Core
Built, run and supported by SwarmioGPU orchestration, operators, customers, users, applications, placement, identity and audit, held in one place and driven onto every cluster. Runs GPU as a Service on its own.
- Tenancy
- Provisioning
- Catalogue
- Identity
- Audit
Infrastructure
Provided by Swarmio or by your organizationServers, GPUs, storage and network beneath a dedicated Kubernetes cluster. Nothing proprietary is required from the hardware beneath.
- Kubernetes
- NVIDIA GPU Operator
- Hardware from any vendor
Architecture
Five propertiesset it apart.
Seven principles govern every design decision, and each is tested when a deployment is accepted. These are the properties they produce.
- 01
Governance compiled into infrastructure.
Core holds who may use what, where it runs and what it may spend, and turns that into running deployments. A new application goes live for a team with no ticket and no engineer.
- 02
One control plane, many sites.
A single control plane places workloads across data centres, in-country clouds and edge locations, under one catalogue and one console.
- 03
Sovereignty by placement.
Every customer is held to an allow-list of sites and model gateways, and every application instance is bound to its gateway for life.
- 04
Metering that reaches billing.
Model usage and infrastructure use (GPU, CPU, memory, storage and network) are attributed to the customer that consumed them and exported to billing.
- 05
Standards only.
Kubernetes, OpenID Connect, an OpenAI-compatible model API, S3 and standard metrics. Nothing proprietary is required from the hardware beneath.
Deployment
One control plane.Many sites.
Core decides what runs where; each site runs it, and application data stays at the site. Hosted runs on Swarmio capacity. Dedicated runs on your own GPU clusters, with a control plane dedicated to you that can run in your jurisdiction.
Swarmio Core
The control plane, with the Swarmio Conductor model gateway. Core’s own services use no GPU, so the full GPU inventory stays available for work.
- API and portal
- Identity
- Secrets
- Declared state
- Audit
- Site A
Sovereign data centre
National-scale capacity for government and regulated workloads.
- Site B
In-country cloud
Capacity from a cloud provider inside the country.
- Site C
Edge or on-premises
In your own facility or at the network edge, close to the data.
- Applications, one instance per customer
- Models on the site’s GPUs
- A Kubernetes cluster
- Application data stays at the site
A new site joins under the same control plane, customers and catalogue, with no second platform and no second portal.
Inside the platform
Built by Swarmio,from the control plane up.
Three modules hold the platform together, and the GPUs beneath them are offered at four levels of isolation from the same catalogue.
- Control plane
Swarmio Core
Holds who the operators and customers are, what they may use and where it runs, and drives every cluster to match.
- Platform, operator, customer, instance: a customer has no visibility of, or path to, any other
- Provisioning without a ticket, and nothing left half-built if a step fails
- A REST API with OAuth 2.0, authorized on every call, with the console a pure client of it
- An append-only audit record streamed to your security monitoring, with request bodies never captured
- Model gateway
Swarmio Conductor
One OpenAI-compatible endpoint in front of every model you offer. An existing application moves onto governed models by changing two values: a base address and a key.
- Keys per application instance or user, revocable one by one and stored only as hashes
- Budgets per customer and per key, checked before every request
- Rate limits, weighted routing and failover to the next healthy model
- Local-only by default: an external provider is added only on your written approval
- Attribution
Swarmio Meter
What did each customer consume, on which resources, and what did it cost? Every unit is assigned to a paying customer at a defensible rate.
- Model use: requests, input and output tokens, cost and latency
- GPU allocation over time, plus CPU, memory, storage and network
- A per-customer usage export and a monthly snapshot for reconciliation
- Invoicing, tax and collection stay in your own billing system
- GPUs and GPU as a Service
Four levels of isolation
From a whole dedicated device to tokens through Conductor, all launched from the same catalogue and governed by the same control plane.
- Dedicated: one or more whole GPUs
- Partitioned: a hardware partition of a GPU, with its own memory and compute
- Shared: a time-sliced share, suited to development and low-sensitivity work
- Serverless: model access by the token, through Conductor
Security and governance
Governance is enforced,not documented.
Every control has a named enforcement point, and isolation is verified by test when a deployment is accepted, not assumed from configuration.
| Governance question | Control | Enforced in |
|---|---|---|
| Who may use which application? | Catalogue curation per operator; operator and customer roles | Swarmio Core |
| Which models may a customer reach? | Default models per operator, with exceptions per customer | Swarmio Core |
| How much may a customer spend? | Budgets per customer and key; requests are refused once spend reaches the cap | Swarmio Conductor |
| Where do workloads and models run? | Site and gateway allow-lists, region enforced at placement, permanent instance binding | Swarmio Core |
| May external model providers be used at all? | Local-only operation; external-model fallback off by default | Swarmio Conductor |
| What changed, and who changed it? | Append-only audit record, streamed to security monitoring | Swarmio Core |
| What may an agent run? | Commands run in the agent’s own container, with a dangerous-command check | The agent’s container |
| Layer | Mechanism |
|---|---|
| Workload | One Kubernetes namespace per customer |
| Compute | Resource quotas and limit ranges per customer namespace |
| Network, in the cluster | Default-deny network policy per customer namespace, verified by test |
| Network, across sites | Mesh access policy per instance group; no path between customers |
| GPU | Dedicated devices or hardware partitions; time-slicing only on the shared tier |
| Storage | Volumes scoped to the instance, inside the customer’s namespace |
| Model access | Separate gateway team, keys and budget per customer |
| Secrets | A per-instance secret path, generated at launch and delivered into the instance’s namespace |
In transit: TLS at every application address, and an encrypted mesh between the control plane and every site. At rest: customer volumes are encrypted. The gateway’s usage log keeps no prompt or response content.
Open standards
Open interfaces in.Usable data out.
Swarmio works alongside the platforms you already run. No particular server, CPU, network, storage vendor or hypervisor is required.
- Orchestration
Kubernetes API, CNCF-conformant
- GPU access and telemetry
NVIDIA GPU Operator
- Identity
OpenID Connect and OAuth 2.0
- Model API
OpenAI-compatible REST, with streaming
- Storage
Kubernetes CSI and S3-compatible object storage
- Ingress
Kubernetes Gateway API
- Metrics
A standard metrics exposition format
- Audit
A JSON event stream into your SIEM
Adding GPUs, nodes, customers, applications or sites changes capacity or configuration, never the architecture.
Research & patents
Research before product.Patented. Peer-reviewed.
Swarmio’s engineers and university partners have worked on a core problem of distributed computing: deciding, in real time, which compute should serve which user across many sites, within latency and capacity limits. That work earned a granted U.S. patent and peer-reviewed research in reinforcement learning and optimization.
- U.S. patent · Granted 13 July 2021
US 11,063,881 B1
Methods and apparatus for network delay and distance estimation, computing resource selection, and related techniques
A trained model estimates the delay or distance between a user and every candidate compute resource. A Q-learning algorithm then selects the resource, weighing each resource’s capacity and the latency of the users it already serves.
Assignee: Swarmio Inc. · Inventors: K. Vijayasuganthan, S. Stoian, S. Shirmohammadi, S. Mohammed, A. E. Alchalabi · International application WO 2022/090803
View the patent - Peer-reviewed journal · Vol. 2, No. 6 · December 2021
IEEE Transactions on Artificial Intelligence
Fair Server Selection in Edge Computing With Q-Value-Normalized Action-Suppressed Quadruple Q-Learning
A reinforcement-learning method that meets an application’s latency threshold while keeping latency fair across every user in a session. In a cloud-gaming evaluation against existing methods, it reduced the standard deviation of users’ latency by up to 35%.
A. E. Alchalabi, S. Shirmohammadi, S. Mohammed, S. Stoian, K. Vijayasuganthan · pp. 519–527 · doi:10.1109/TAI.2021.3105087
Read on IEEE Xplore - Research partnership
University of Ottawa
DISCOVER Lab, led by Professor Shervin Shirmohammadi, IEEE Fellow
Swarmio’s research was carried out with the University of Ottawa, with support from the Natural Sciences and Engineering Research Council of Canada (NSERC). The partnership’s work on AI-based network latency estimation was also published in IEEE Transactions on Instrumentation and Measurement.
S. A. Mohammed, S. Shirmohammadi, S. Altamimi, “A Multimodal Deep Learning-Based Distributed Network Latency Measurement System,” IEEE Transactions on Instrumentation and Measurement, 2020 · doi:10.1109/TIM.2020.2967877
Read the 2020 paper
What’s next
Published as plainlyas what runs today.
These capabilities are prioritized and in active development. Everything above this section runs in production today.
- Agents and tools
Governed tools and agent-to-agent calls
A tool gateway on the Model Context Protocol with per-customer tool permissions; an agent-to-agent gateway with permissions and iteration budgets; approval of AI Coworkers’ business actions before they run; and a tamper-resistant record of every agent action, kept outside the agent.
- Deployment
Swarmio Sovereign AI Air-gapped
The control plane inside your isolated network, a model gateway at each site that keeps model requests and responses at the site, and model pools pinned per site.
- Gateway
Tighter controls at the gateway
Default budget caps and rate limits on every new customer, and semantic caching as a customer option.
- Identity and integration
Your directory, your order systems
Multi-factor authentication for administrators, federation of your own directory, and server-to-server provisioning from order systems.
- Lifecycle
Change in place
Resize, stop, start, restart and upgrade in place, and a tested customer offboarding procedure with a written deletion list.
How to evaluate Swarmio
On your own workloads,not on slides.
Three steps, ending in a clear decision.
- 01Architecture sessionA working session with your architects on deployment model, sites, identity, billing and security monitoring.
- 02Detailed documentation under NDAThe Technical Architecture paper, the Security and Governance Pack and the Site Readiness requirements, with every capability’s status.
- 03A one-month evaluationOn Swarmio capacity or your own GPUs, against criteria agreed at the start, ending in a clear decision.
The data, the models and the jurisdictionstay yours.
Arrange an architecture session with Swarmio or your local delivery partner, and get the detailed documentation under NDA.