Private AI infrastructure
TensorBreeze
Every GPU you own. One private AI endpoint.
A self-hosted GPU control plane for heterogeneous Windows and Linux machines, with bounded workload routing, fleet telemetry, reservations, and topology-aware orchestration.
Public, sanitized project material.
The problem
Why it exists
Useful GPUs often sit in machines with different operating systems, availability windows, trust levels, and failure modes.
Applications should not need to know which machine owns a device—or become unstable because a remote worker changes underneath them.
- TensorBreeze is currently private while its architecture matures, with an open-source release planned.
- Public diagrams omit real machine names, addresses, credentials, and deployment topology.
Architecture
Operating model
Real-world proofA large model was loaded and executed on a remote RTX 5090 while the originating application remained on another machine.
Capabilities
What it does
Security
Trust model
- Workers initiate outbound connections and expose no general remote shell.
- Read-only telemetry, capacity reservation, and workload execution are separate switches.
- Application credentials select allowlisted models—not nodes, ports, commands, paths, or provider secrets.
Tradeoffs
Key decisions
Stable applications, disposable workers
Applications stay on the reliable side of the boundary while GPU workers can be restarted, upgraded, or taken offline independently.
Capability-specific trust
Telemetry, queueing, routing, and execution do not share one oversized credential or one all-powerful process.
Current state
Current state
- Controller, Worker, Runner, Router, and Tunnel boundaries
- Windows, WSL2, and native Linux worker support
- Read-only telemetry with separately gated controls
- OpenAI-compatible application routing
- Broader application marketplace
- Deeper topology-aware scheduling
- Open-source release once the architecture and boundaries are ready
Lessons
What the work clarified
- Heterogeneous compute is primarily a policy problem, not a device-discovery problem.
- A useful control plane makes absence and degradation first-class states.
- The safest remote control is a small, typed contract—not a shell with good intentions.