My Self-Sovereign / Local / Private / Secure LLM Setup我的主权 AI：本地、私密、安全的 LLM 配置

The Setup

We are living through the agent transition: AI has moved from “chatbot answers your question” to “agent takes a task and uses hundreds of tools to complete it.” OpenClaw has been central to this shift. But Vitalik’s read of the security culture around this space is damning — roughly 15% of skills contain malicious instructions, agents can modify their own system prompts without confirmation, and a single malicious webpage can trigger arbitrary code execution.

His response is principled: build a setup where privacy, security, and self-sovereignty are non-negotiable from day one.

Key Takeaways

• All inference runs locally. Vitalik uses Qwen3.5:35B on either an NVIDIA 5090 laptop (90 tok/sec) or AMD Ryzen AI Max Pro with 128GB unified memory (51 tok/sec). The DGX Spark — despite NVIDIA’s marketing — underperforms both. His floor: anything below 50 tok/sec is too annoying for daily use.

• llama-server, not Ollama. It can fit larger models in GPU memory (Ollama couldn’t run Qwen3.5:35B on his hardware), and exposes a local HTTP endpoint any OpenAI-compatible client can point to — including Claude Code.

• Sandboxing is mandatory. He uses bubblewrap to create filesystem sandboxes rooted in specific directories. Agents inside can only see whitelisted files and ports. This is the structural defense against prompt injection attacks from malicious web content.

• Agent permissions are explicit. His custom messaging daemon only allows the LLM to send messages to himself freely; sending to others requires human confirmation. No wallet moves without sign-off.

• Local world knowledge reduces internet exposure. A 1TB Wikipedia dump plus local manuals means fewer search queries leaking to external services. He wants someone to build a Tor-wrapped search skill for the remaining queries.

• Small models still fail on hard tasks. Qwen3.5:35B one-shots simple apps and games, but struggles with domain-specific cryptographic implementations. His honest concession: he still routes hard problems to Claude.

Why It Matters

The mainstream framing around AI agents treats privacy as optional and security as someone else’s problem. Vitalik is arguing the opposite: as agents gain real-world capabilities — crypto transactions, messaging, file access — the cost of complacency becomes catastrophic. The self-sovereign stack he describes isn’t paranoia. It’s the natural consequence of taking seriously what these tools can actually do.

For anyone running AI agents with access to financial assets or private communications, this is a direct challenge: have you designed for containment, or are you trusting the ecosystem to not be malicious?

What to watch:

• Local LLM performance curves (AMD vs. NVIDIA convergence matters for non-CUDA users)
• OpenClaw and similar agents adding mandatory human confirmation layers for high-risk actions
• Whether the “Tor-wrapped search skill” Vitalik flags as missing gets built
• Regulatory/institutional pressure on cloud AI providers around data retention