All projects
Physical AISoftware & Agents
NurseArm
An LLM agent that safely drives an SO-101 arm for medication tasks through bounded MCP skills.

Overview
NurseArm connects an SO-101 robot arm to a browser interface and an LLM agent. The agent never produces raw motor commands. It discovers a bounded set of MCP tools, picks a registered skill, executes it and reports a structured result. Every decision lands in an append-only audit log.
Skills include direct Cartesian primitives and two trained ACT policies: sorting pills and handing over a green or black pill when MediaPipe detects an open, palm-up hand. A calendar can trigger the right skill at medication time. Text, voice (Faster-Whisper) and an optional Telegram bridge are all supported inputs.
Highlights
- 01Bounded MCP skill server: the LLM selects tools and never improvises motor commands.
- 02ACT policies (sort_pills, handover_pill) deployed through lerobot-rollout.
- 03Hand openness and palm-up detection with MediaPipe on RealSense or webcam streams.
- 04Local-first agent (Ollama + Qwen 3) with Claude as a drop-in backend.
- 05An honesty document that separates what was built, what was borrowed and what was verified.