Article
What "AI controls your phone" actually requires: observation paths, input paths, and verification
NatalieY
• Natalie Y PRO
NatalieY
AI & ML interests
AI agents, edge AI, on-device LLM, embedded systems, autonomous agents
Recent Activity
published an article 28 days ago
Grounding an agent's answers about its own environment: a small but useful patternOrganizations
published an article 3 days ago
posted an update 28 days ago
published an article 28 days ago
Article
Grounding an agent's answers about its own environment: a small but useful pattern
NatalieY
• posted an update about 1 month ago
Post
68
Shipped a full-duplex agent architecture for Aiden: real-time voice model handles the live conversation, a separate stronger model handles device-control tasks in the background, coordinated async through a task queue instead of one shared loop.
Grounded in full-duplex speech research (Moshi's parallel-stream model, dual-tower dialogue modeling), not just voice bolted onto an existing agent pipeline.
A few design choices worth sharing: task "completed" and "successful" are tracked as separate signals so the agent never reports a stalled task as done. Backend results get a 500ms aggregation window before reaching the foreground, so closely-timed results don't trigger competing responses. Device tasks run strictly one at a time: ambiguous screen/input ownership wasn't worth the complexity of parallelizing.
Dev-board stage, not validated across every phone/board/audio setup yet. Full writeup: https://aidenai.io/blog/when-voice-meets-the-physical-world-inside-aidens-full-duplex-agent-architecture/
Curious how others here have handled foreground/background coordination when combining real-time voice with a separate task-execution model.
Grounded in full-duplex speech research (Moshi's parallel-stream model, dual-tower dialogue modeling), not just voice bolted onto an existing agent pipeline.
A few design choices worth sharing: task "completed" and "successful" are tracked as separate signals so the agent never reports a stalled task as done. Backend results get a 500ms aggregation window before reaching the foreground, so closely-timed results don't trigger competing responses. Device tasks run strictly one at a time: ambiguous screen/input ownership wasn't worth the complexity of parallelizing.
Dev-board stage, not validated across every phone/board/audio setup yet. Full writeup: https://aidenai.io/blog/when-voice-meets-the-physical-world-inside-aidens-full-duplex-agent-architecture/
Curious how others here have handled foreground/background coordination when combining real-time voice with a separate task-execution model.
upvoted an article about 1 month ago
Article
Coordinating a real-time voice model with a separate task-execution agent: a practical architecture
NatalieY
• • 3 published an article about 1 month ago
Article
Coordinating a real-time voice model with a separate task-execution agent: a practical architecture
NatalieY
• • 3 posted an update about 1 month ago
Post
105
Sharing a small but instructive fix from Aiden's firmware. Before PR #586, giving the agent a remote skill URL didn't reliably route to an installation action: it could just as easily be read as a general content-retrieval request, with the agent reaching for a web scraper, shelling out to curl, creating files manually, and patching them in step by step. Every individual tool call worked. The composition was still the wrong shape for a request with one clear, singular intent.
The fix gives skill installation its own dedicated action. A supported HTTP(S), GitHub tree, or GitHub blob Skill URL now routes to skill_manage install, which stages SKILL.md and companion files, validates the staged content against supported requirements, and only then publishes atomically, never exposing partial files mid-install.
https://huggingface.co/blog/NatalieY/aiden-can-now-install-ai-agent-skills-directly
The fix gives skill installation its own dedicated action. A supported HTTP(S), GitHub tree, or GitHub blob Skill URL now routes to skill_manage install, which stages SKILL.md and companion files, validates the staged content against supported requirements, and only then publishes atomically, never exposing partial files mid-install.
https://huggingface.co/blog/NatalieY/aiden-can-now-install-ai-agent-skills-directly
published an article about 1 month ago
Article
Routing intent correctly: a case study in why "the agent can technically do it" isn't the same as "the agent should do it this way"
NatalieY
• reacted to their post with 🔥 about 2 months ago
Post
1144
👋 Quick update on what shipped in Aiden's firmware this week, it is mostly developer-experience stuff, but genuinely useful if you're building against the agent:
Native Anthropic support: pick Claude models right in the config page now, streaming + tool calls included
No hardware? No problem: new Docker sandbox lets you run Config Web + Agent Web without a dev board, connects to MobileGym/ADB
Two HDMI bridge chips supported (RK628D + TC358743), auto-detected, RK628D gets you up to 1080p60
Python packages now persist across sessions instead of disappearing every restart
Provider switching got smarter: remembers your last model, less manual config-syncing between LLM/TTS/STT
None of these are new agent behaviors, just a lot less friction if you're actually building. Full technical breakdown here if you want the details: https://huggingface.co/blog/NatalieY/aiden-updates
Has anyone tried the Docker sandbox path yet?
AidenAgent
Native Anthropic support: pick Claude models right in the config page now, streaming + tool calls included
No hardware? No problem: new Docker sandbox lets you run Config Web + Agent Web without a dev board, connects to MobileGym/ADB
Two HDMI bridge chips supported (RK628D + TC358743), auto-detected, RK628D gets you up to 1080p60
Python packages now persist across sessions instead of disappearing every restart
Provider switching got smarter: remembers your last model, less manual config-syncing between LLM/TTS/STT
None of these are new agent behaviors, just a lot less friction if you're actually building. Full technical breakdown here if you want the details: https://huggingface.co/blog/NatalieY/aiden-updates
Has anyone tried the Docker sandbox path yet?
- 2 replies
published an article about 2 months ago
Article
Recovering Long-Running Agent Tasks From Context Overflow, Without Blind Restarts
AidenAgent
• posted an update about 2 months ago
Post
1144
👋 Quick update on what shipped in Aiden's firmware this week, it is mostly developer-experience stuff, but genuinely useful if you're building against the agent:
Native Anthropic support: pick Claude models right in the config page now, streaming + tool calls included
No hardware? No problem: new Docker sandbox lets you run Config Web + Agent Web without a dev board, connects to MobileGym/ADB
Two HDMI bridge chips supported (RK628D + TC358743), auto-detected, RK628D gets you up to 1080p60
Python packages now persist across sessions instead of disappearing every restart
Provider switching got smarter: remembers your last model, less manual config-syncing between LLM/TTS/STT
None of these are new agent behaviors, just a lot less friction if you're actually building. Full technical breakdown here if you want the details: https://huggingface.co/blog/NatalieY/aiden-updates
Has anyone tried the Docker sandbox path yet?
AidenAgent
Native Anthropic support: pick Claude models right in the config page now, streaming + tool calls included
No hardware? No problem: new Docker sandbox lets you run Config Web + Agent Web without a dev board, connects to MobileGym/ADB
Two HDMI bridge chips supported (RK628D + TC358743), auto-detected, RK628D gets you up to 1080p60
Python packages now persist across sessions instead of disappearing every restart
Provider switching got smarter: remembers your last model, less manual config-syncing between LLM/TTS/STT
None of these are new agent behaviors, just a lot less friction if you're actually building. Full technical breakdown here if you want the details: https://huggingface.co/blog/NatalieY/aiden-updates
Has anyone tried the Docker sandbox path yet?
- 2 replies
published an article about 2 months ago
Article
This Week in Aiden Firmware: Making the Agent Easier to Actually Develop Against
NatalieY
• liked a
Space about 2 months ago
published an article about 2 months ago
Article
Two Small-Hardware AI Architectures: Managed Cloud Pipeline vs. Bring-Your-Own-Model
NatalieY
• upvoted an article about 2 months ago
Article
Where Model-Driven Screen Interaction Actually Runs Out of Room
NatalieY
• • 1 published an article about 2 months ago
Article
Where Model-Driven Screen Interaction Actually Runs Out of Room
NatalieY
• • 1 reacted to their post with 🔥 2 months ago
Post
2057
Spent a while chasing a genuinely strange iOS bug while building Aiden (a physical agent that drives phones over USB HID): modifier-key shortcuts like Cmd+V would silently fail while plain keystrokes worked fine every time.
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
posted an update 2 months ago
Post
2057
Spent a while chasing a genuinely strange iOS bug while building Aiden (a physical agent that drives phones over USB HID): modifier-key shortcuts like Cmd+V would silently fail while plain keystrokes worked fine every time.
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
Turned out iOS was routing the command to the wrong process (SpringBoard, not the actual foreground app) whenever a keyboard and mouse were both present at the same time as AssistiveTouch. Confirmed it wasn't specific to our hardware, reproduced it on a completely unrelated gaming keyboard and a Bluetooth keyboard too.
Full writeup with the actual experiment table and log output: https://huggingface.co/blog/NatalieY/debugging-aiden
Curious if anyone here has hit this same failure mode building on iOS accessibility APIs.
Repo: https://github.com/AidenAI-IO/aiden-firmware
upvoted an article 3 months ago
Article
Aiden: a physical AI agent that controls phones over USB HID
AidenAgent
• • 1 published an article 3 months ago
Article
Aiden: a physical AI agent that controls phones over USB HID
AidenAgent
• • 1