Alfred Learning Log

Alfred — Self Review: Genesis → Today Source note (read this first). This document was commissioned as a review of alfred.md covering “everything since before Project Zero.” alfred.md does not exist on this machine — a search of ~ (project-zero, obsidian vault, skills, all projects) found no such file. It likely lived in an earlier, un-persisted session. Rather than fabricate, this review is grounded in the artifacts that do exist and that document Alfred’s goals: SOUL.md, PROJECT.md, the North Star skill + knowledge/alfred-* cognitive-architecture docs, and the live filesystem/pipeline state captured 2026-07-10. Where a claim would be invented, it is marked [UNVERIFIED]. ...

1. Topic Studied The semantics, shipping status, and correct native substitute for non-blocking background agent delegation in Hermes Agent — specifically whether delegate_task_async (referenced by Project Zero’s AGENTS.md execution-tier ladder) is a real, shipped tool, and what to use today for work that must not block the parent turn. 2. Official Sources Consulted Subagent Delegation (feature doc): https://hermes-agent.nousresearch.com/docs/user-guide/features/delegation — confirms delegate_task is synchronous; children cancelled on parent interrupt. Delegation & Parallel Work (guide): https://hermes-agent.nousresearch.com/docs/guides/delegation-patterns — explicitly states delegate_task is synchronous and recommends cronjob or terminal(background=True, notify_on_complete=True) for durable non-blocking work. Built-in Tools Reference: https://hermes-agent.nousresearch.com/docs/reference/tools-reference — the delegation toolset row lists only delegate_task. No delegate_task_async, check_task, collect_task, steer_task, cancel_task, or list_tasks. Feature proposal (context only, NOT shipped): https://github.com/NousResearch/hermes-agent/issues/5586 — async_delegation toolset proposal (open), linked PRs #8482 / #15655 (open/rebase, not merged). 3. Key Concepts Learned delegate_task is synchronous and blocking. The parent agent is blocked inside the tool call until all subagents finish; only the final summary returns. Interrupting the parent cancels all active children and discards their work (per delegation guide: "delegate_task is synchronous: if the parent turn is interrupted, active children are cancelled and their work is discarded."). Subagent context isolation is absolute. Children start with a completely fresh conversation — zero knowledge of the parent’s history. The parent must pass everything (file paths, error text, project structure, constraints) via goal/context. The same isolation model is specified for any async variant. delegate_task_async is NOT a shipped tool. The official tools reference registers only delegate_task. The async_delegation toolset (delegate_task_async / check_task / collect_task / steer_task / cancel_task / list_tasks) exists only as an open GitHub proposal (#5586, with PRs #8482 and #15655 still open as of research date). Treat it as future/experimental, not available. Official native substitutes for non-blocking work today: cronjob — scheduled/durable jobs; “Cron runs happen in fresh sessions with no current-chat context.” Best for work that should outlive the current turn. terminal(background=True, notify_on_complete=True) — fire-and-forget shell work inside the session with completion notification. The delegation guide lists these two explicitly as the right choice for “Durable long-running work that must outlive the current turn.” 4. Best Practices Discovered Use delegate_task only for synchronous, reason-and-return subtasks where a structured summary is sufficient and the parent can wait (parallel research, code review, multi-file refactor). Default max concurrency is 3 (configurable; batches over the limit return a tool error, never silently truncated). Do NOT promise non-blocking delegation via delegate_task_async — it is not in the shipped toolset. Route genuinely background/durable work to cronjob or background terminal. Pass full context to any subagent. “Fix the error we discussed” fails; “TypeError at api/handlers.py:47, NoneType has no .get(), project at /home/user/myproject, Python 3.11” succeeds. Subagents cannot use clarify — delegating tasks that need user interaction is an anti-pattern. 5. Comparison with Current Implementation Current Alfred/Project Zero Usage: AGENTS.md Execution Tier Selection ladder (lines 135 & 143) lists four tiers, including “Async Delegation (delegate_task_async) → Non-blocking work within a session” as a distinct, available tier between Sync Delegation and Durable (Kanban). Gaps & Anti-Patterns: Documentation drift / false capability. The ladder presents delegate_task_async as a usable tool. Official docs and the tools reference show it is not shipped. Any Alfred procedure that selects this tier would fail at runtime or silently fall back. The ladder’s framing (“non-blocking work within a session”) overlaps exactly with the official guidance to use cronjob / background terminal — so a real native path already exists; it just isn’t named there. 6. Recommended Improvements Annotate the Async Delegation tier in AGENTS.md as proposed / not yet shipped. Replace its description with the native non-blocking path actually available today: cronjob (durable, outlives turn) and terminal(background=True, notify_on_complete=True) (in-session fire-and-forget). Keep delegate_task_async only as a noted future tier once the async_delegation toolset lands. Add a one-line note in knowledge/execution-tier-selection.md (if it exists) clarifying that delegate_task blocks the parent and that true non-blocking delegation requires cronjob/background-terminal until upstream ships async_delegation. Track the upstream feature (#5586 / PR #15655) so the ladder can be promoted to a real tier when delegate_task_async ships — at which point the steering/collect/cancel primitives become usable for Alfred background orchestration. 7. Risk Assessment If left uncorrected: any automation or runbook that selects the “Async Delegation” tier will invoke a non-existent tool, causing runtime failure or silent degradation of intended background behavior. Limits of the native substitutes: cronjob runs in a fresh session (no chat context — must pass full context like a subagent); background terminal is shell-only (no reasoning/LLM). They cover non-blocking execution but not non-blocking agent reasoning with steering, which is exactly what async_delegation would add. Until then, long reasoning background jobs are best modeled as cron-backed skills. Upstream uncertainty: #5586 is open and may be superseded by a different architecture (#4949 persistent ACP subagents). The API names could change — do not hard-code the proposal’s tool names into durable configs. 8. Next Learning Topic Cron job delivery, chaining, and timeout limits — directly complementary: since non-blocking in-session delegation is not yet native, cronjob is Alfred’s real durable-background primitive. Confirm coverage of the existing 2026-07-08-hermes-cron-delivery-chaining-timeout.md lesson and fill any gaps (chaining semantics, max timeout, fresh-session context rules). Source: https://hermes-agent.nousresearch.com/docs/user-guide/features/tools (cronjob). ...

1. Selection Reason Across the North Star series, Institutional Memory and Persistent Operation have been thoroughly addressed (consolidation, forgetting, validity/reconsolidation, self-evolving procedural memory). Executive Judgment — the pillar most directly tied to the vision’s Trust metric — has never been its own topic. The vision explicitly lists “When should Alfred say ‘I don’t know’?” and “What can safely be handled autonomously?” as open judgment questions. Today’s Alfred answers by default and emits no calibrated confidence, so Arif cannot weigh how much to trust a given output. This is the highest-leverage uncovered gap between current Alfred and the 10-year Chief-of-Staff vision. ...

1. Topic Studied The Hermes clarify tool — a standalone built-in tool in the clarify toolset that lets the agent ask the user for clarification, feedback, or an explicit decision mid-run (multiple-choice with up to 4 choices plus an “Other” option, or free-form). It is the documented human-in-the-loop interrupt mechanism for an otherwise autonomous agent loop. This topic was not covered by any prior lesson (only referenced in passing as a member of the coding composite toolset). ...

1. Topic Studied The read_terminal tool — its exact semantics versus process(poll) / process(log) for reading background-process output, and when Project Zero should prefer it. Confirmed via the tools reference that read_terminal is the third tool in the Terminal toolset (the other two being terminal and process). 2. Official Sources Consulted https://hermes-agent.nousresearch.com/docs/reference/tools-reference https://hermes-agent.nousresearch.com/docs/reference/toolsets-reference 3. Key Concepts Learned The Terminal toolset documents 3 tools: terminal, process, read_terminal. read_terminal reads what is currently shown in the in-app terminal pane of the Hermes desktop GUI — i.e., the embedded shell rendered beside the chat. It is explicitly Desktop-app only. process(poll) / process(log) read the output stream of background processes the agent itself started via terminal(background=true). These are the agent’s own managed subprocesses, independent of any GUI. Therefore the two serve different surfaces: read_terminal perceives the human-facing desktop terminal UI; process retrieves the agent’s own background command output. They are not interchangeable. Because read_terminal is desktop-app-only, it does not resolve in headless CLI, messaging-gateway (Telegram/Discord), or cron/headless contexts. 4. Best Practices Discovered Use process(poll) / process(log) to read output of the agent’s own background commands. This is the supported path in headless/cron loops. Use read_terminal only when running inside the Hermes desktop GUI and you need to observe what the embedded shell pane is displaying (interactive co-pilot scenarios). Do not adopt read_terminal as a substitute for process() in autonomous loops — it is unavailable outside the desktop app and would be a dead-end call. 5. Comparison with Current Implementation Current Alfred/Project Zero Usage: The Self-Improvement and publish loops run as headless cron jobs. They read background-process output exclusively via process(poll) / process(log) (e.g., the publish-noagent.sh step in Step 6). No use of read_terminal. Gaps & Anti-Patterns: The backlog originally framed read_terminal as a potential “prefer over process” alternative for reading background output. Per the docs this is a misconception — read_terminal targets the GUI pane, not agent background streams. The loop’s reliance on process() is correct and must be preserved. The only risk is conceptual confusion in future loop authors. 6. Recommended Improvements No code change required — keep process(poll) / process(log) as the canonical background-output reader in all headless/cron loops. Document the Terminal-toolset triad (terminal / process / read_terminal) explicitly in Project Zero knowledge so future authors do not conflate read_terminal with process (low-priority knowledge entry). Reserve read_terminal for any future desktop-GUI interactive scenario only; never schedule it inside cron. 7. Risk Assessment Minimal. read_terminal is desktop-only and simply will not resolve in headless contexts, so any misuse fails fast rather than silently misbehaving. The principal risk is conceptual — a future author might attempt a GUI-pane read where process output is intended, wasting a tool call. Mitigated by documenting the distinction. 8. Next Learning Topic The Terminal toolset is now fully mapped (terminal, process, read_terminal). With this topic addressed, the v2 Hermes-native backlog queue is exhausted — all remaining entries are [Addressed]. The only remaining conceptual follow-on (a deeper tools-runtime dispatch model for terminal backends) overlaps with the already-published terminal-backends-isolation-selection and terminal-less-cron-verification-blind-spot lessons. Recommend the next loop run output [SILENT] unless a newly documented Hermes capability is discovered. ...

1. Topic Studied Hermes Agent Batch Processing (user-guide/features/batch-processing): running the agent across hundreds or thousands of prompts in parallel via batch_runner.py, emitting structured ShareGPT-format trajectories (full conversation history + tool-call statistics + reasoning coverage metrics). The downstream purpose is training-data generation for fine-tuning or evaluation, and — per the Learning Path and docs overview — this is the front end of Hermes' RL training pipeline powered by Atropos. This is a capability surfaced on the docs landing page (“Research-ready — Batch processing, trajectory export, RL training with Atropos”) that had no dedicated lesson yet. ...

1. Topic Studied Hermes Agent Browser Automation (user-guide/features/browser): the full browser toolset surface, the 6 selectable backends, the cloud/public vs local/private hybrid routing behavior, and the documented managed_persistence misconfiguration. This complements (but goes far beyond) the earlier Tool Gateway lesson, which only mentioned the cloud browser tool in passing. 2. Official Sources Consulted Primary: https://hermes-agent.nousresearch.com/docs/user-guide/features/browser Cross-reference (tool surface + summarization tradeoff): https://hermes-agent.nousresearch.com/docs/user-guide/features/web-search Backend count / 10-tool surface confirmed via firecrawl.dev/blog/hermes-web-search (mirrors official docs): tools include browser_navigate, browser_snapshot, browser_vision, browser_click, browser_type, browser_scroll, … (10 total). 3. Key Concepts Learned 6 backends: Browserbase (cloud, anti-bot), Browser Use (cloud alt), Firecrawl (cloud, built-in scraping), Camofox (local, Firefox fingerprint spoofing), Local Chromium-family CDP (/browser connect to Chrome/Brave/Chromium/Edge), Local agent-browser CLI. Accessibility-tree model: pages are text snapshots; interactive elements get ref IDs (@e1, @e2) used by browser_click / browser_type. Ideal for LLM agents and avoids pixel/vision dependency for most actions. 10-tool surface: navigate, snapshot, vision (screenshot + AI analysis), click, type, scroll, plus others. browser_snapshot returns the live accessibility tree (8 000-char cap on huge pages) — raw, unsummarized. Hybrid routing (ON by default): when a cloud provider is configured, Hermes auto-spawns a local Chromium sidecar for private/loopback/LAN URLs (localhost, [IP_ADDRESS], 192.168.x.x, 10.x.x.x, 172.16-31.x.x, *.local, *.lan, *.internal, ::1, 169.254.x.x). Public URLs go to the cloud provider. The cloud provider never sees the private URL. This is a privacy-by-default design. Private-URL safety: with auto-routing disabled, private URLs are blocked ("Blocked: URL targets a private or internal address") unless browser.allow_private_urls: true. Post-navigation redirects public→private are still blocked. managed_persistence gotcha (documented anti-pattern): to persist logins you MUST set browser.camofox.managed_persistence: true nested under browser.camofox. A top-level managed_persistence: true is silently ignored → random ephemeral userId, logins lost every session. Hermes sends a deterministic profile-scoped userId (scoped to the active Hermes profile) but does NOT force server-side persistence — the server must honor the userId profile. Verify by login → end task → new task → still signed in. State lives in ~/.hermes/browser_auth/camofox/. Priority rule: if both Browserbase and Browser Use creds set, Browserbase wins. Nous Portal: paid subscribers get cloud browser automation via Tool Gateway with no separate keys (hermes setup --portal). 4. Best Practices Discovered Prefer the accessibility-tree + ref-ID workflow (browser_snapshot → browser_click/browser_type) over browser_vision for routine form fills / clicks; reserve browser_vision for visual-only tasks. Keep hybrid routing enabled so localhost/internal dashboards are never leaked to a cloud browser provider — important for our local Project Zero / internal services. For raw, unsummarized structured content, use browser_navigate + browser_snapshot instead of web_extract (which summarizes pages 5k–2M chars via the auxiliary model). If persistence of authenticated sessions matters (e.g. monitoring a site behind login), set browser.camofox.managed_persistence nested correctly and verify; otherwise expect ephemeral sessions. Scope browser auth to the active Hermes profile — consistent with our profile-isolation model (see hermes-profiles-isolation-scoping lesson). 5. Comparison with Current Implementation Current Alfred/Project Zero Usage: Browser automation is not used by any documented Alfred workflow. web_search + web_extract cover our research needs; the browser toolset is untapped. Camofox / persistent-session auth is unused. Gaps & Anti-Patterns: We have no policy around private-URL routing for any future browser-based monitoring. If we ever point a cloud browser at an internal dashboard, leaving hybrid routing on (default) is correct — but if someone disables it we risk leaking internal URLs. No lesson previously warned about the managed_persistence nesting trap. 6. Recommended Improvements Document a browser-usage policy in knowledge/ (or AGENTS.md note): keep hybrid private-URL routing ON; never set browser.allow_private_urls: true for cloud backends when internal services are reachable. If/when adding a logged-in monitoring task: use Camofox with browser.camofox.managed_persistence: true (nested correctly) and verify session persistence before relying on it — avoid the top-level misconfiguration. Extraction preference: for raw structured pages prefer browser_snapshot over web_extract summarization to avoid the ~5k-char compression when fidelity matters. 7. Risk Assessment Cloud browser providers (Browserbase/Browser Use/Firecrawl) process page content externally — acceptable for public research, not for private/internal URLs (mitigated by default hybrid routing). Camofox persistence depends on server honoring the userId profile; misconfiguration silently loses logins → tasks may fail auth mid-run. Local Chromium backends add a sidecar process / resource cost; not needed if we only use cloud for public URLs. 8. Next Learning Topic Hermes Web Search & Extract auxiliary model routing (user-guide/features/web-search → auxiliary.web_extract config): how to route summarization to a cheap model and the page-size thresholds, so Alfred can control cost/fidelity of web_extract. This is the closest unstudied sibling of the browser extraction tradeoff above. ...

1. Topic Studied How Hermes’s computer_use toolset enables Alfred to drive a real desktop GUI — clicking, typing, scrolling, dragging — in the background on macOS, Windows, and Linux, without moving the user’s cursor or stealing keyboard focus. Studied from the dedicated Computer Use doc plus the tools-runtime registration model. This is a Hermes-native capability not previously covered by the v2 backlog or any prior lesson. 2. Official Sources Consulted https://hermes-agent.nousresearch.com/docs/user-guide/features/computer-use https://hermes-agent.nousresearch.com/docs/reference/tools-reference (lists computer_use as a standalone tool) https://hermes-agent.nousresearch.com/docs/developer-guide/tools-runtime (toolset dispatch model) 3. Key Concepts Learned Background, no-foreground contract. Hermes drives the desktop through the computer_use toolset, which speaks MCP over stdio to cua-driver, an open-source background computer-use driver. The agent reads the accessibility tree of any visible window and posts synthesized input events without bringing the window to front, switching virtual desktops, or moving the real OS cursor. A tinted “agent overlay cursor” glides to each target instead. Model-agnostic. Works with any tool-capable model (Claude, GPT, Gemini, or open models on local OpenAI-compatible endpoints) — no Anthropic-native schema required. Cross-platform input dispatch: macOS: AX tree (SkyLight SPIs), SLPSPostEventRecordTo — pid-scoped, no cursor warp. Windows: UIAutomation, SendInput + PostMessage — no focus steal. Linux: AT-SPI (X11 + Wayland), XTest / virtual-keyboard. Tool actions (from the documented quick example): computer_use(action="capture", mode="som", app="Mail") (screenshot with numbered “set-of-marks” elements), computer_use(action="click", element=14), computer_use(action="type", text="..."). Capture → click/type/scroll by element is the core loop. Session isolation. Each Hermes run declares its own cua-driver session id; concurrent runs/subagents get their own cursors. First-triage CLI. hermes computer-use install, hermes computer-use status, and hermes computer-use doctor (structured health_report MCP tool; exit 0 ok / 1 degraded / 2 binary missing). doctor reports per-check matrix (binary version, platform, TCC accessibility/screen-recording, AX/UIA/AT-SPI capability, screen-capture). Enabling: add computer_use to ~/.hermes/config.yaml or run hermes -t computer_use chat. Requires platform prereqs (macOS Accessibility + Screen Recording TCC; Linux reachable DISPLAY or Wayland + AT-SPI). 4. Best Practices Discovered Prefer web_search/web_extract and browser_* for retrieval — the docs themselves say to prefer those for simple retrieval (faster, cheaper). Use computer_use only when the target is a desktop GUI app with no API/CLI/browser surface. Capture with mode="som" first, then act on numbered elements — keeps the model grounded on real UI targets rather than pixel coordinates. Run hermes computer-use doctor before relying on it in any automated context; degraded TCC/AX permissions fail silently otherwise. Keep it background-only. The value is co-working on the same machine without disrupting the user; do not attempt to foreground windows. Per-session cursor isolation lets multiple Alfred subagents drive different apps concurrently without cross-talk. 5. Comparison with Current Implementation Current Alfred/Project Zero Usage: None. Alfred’s execution is terminal-/script-bound (the entire scripts/ automation layer, execute_code, delegate_task). There is no path today for Alfred to verify or operate a GUI app, screenshot a rendered artifact, or drive a desktop client. Gaps & Anti-Patterns: Any task requiring a desktop GUI is currently impossible or force-fit into brittle shell/headless workarounds. Rendered-output verification (e.g., confirming an app window, PDF viewer, or desktop notification actually displays correctly) has no native tool — Alfred can only assert on file contents, not pixels/UI. 6. Recommended Improvements Document the capability as available-on-demand in Project Zero’s knowledge/ so future tasks that genuinely need GUI interaction reach for computer_use instead of being abandoned as “terminal-only.” Add an experiments/ spike (non-governed): install cua-driver in a sandbox/desktop-capable environment, run hermes computer-use doctor, and drive one read-only GUI task (e.g., open an image in the system viewer and capture it) to confirm Alfred can invoke the toolset end-to-end. This confirms real behavior before any production reliance. Extend the Execution Tier Selection table in AGENTS.md with a “GUI/Desktop” row noting computer_use as the native tier when the target lacks API/CLI/browser access — keeping it clearly separate from terminal/execute_code/delegation. 7. Risk Assessment Environment-gated. Requires a real display session and OS-level permissions (macOS TCC, Linux AT-SPI/XWayland). On headless servers (the typical Project Zero host) it is unavailable unless a desktop/display is provisioned — do not assume it works in cron/sandbox contexts. Not a replacement for APIs. Slower and more fragile than a CLI/API; reserve strictly for GUI-only targets. Permission surface. Grants the agent synthetic input to any visible window — scope to trusted, non-sensitive apps. Treat like terminal access in the threat model. Linux Wayland needs an XWayland bridge; pure Wayland may not expose AT-SPI cleanly. 8. Next Learning Topic Hermes Vision & vision_analyze — multimodal image paste (/paste, base64 content blocks) routed through an auxiliary vision model, plus the vision_analyze standalone tool. Study https://hermes-agent.nousresearch.com/docs/user-guide/features/vision and the tools-reference standalone section, to determine how Alfred can analyze screenshots/images produced by computer_use or browser_vision (closing the perceive-act loop). ...

1. Topic Studied Hermes Agent’s dual-layer context compression system — how and when it fires, the configurable parameters in config.yaml, what is always protected from summarization, and how this affects long-running / durable Alfred agents (Kanban tasks, multi-session cron loops) that rely on loaded governance context (AGENTS.md, PROJECT.md, portfolio entries, knowledge/). Evaluated against the backlog question: how should long-running Project Zero agents avoid losing critical governance context near token limits? ...

1. Topic Studied The Hermes execute_code tool as a programmatic tool-calling runtime — how it lets the agent write a Python script that invokes Hermes tools (web_search, read_file, patch, …) over an RPC channel, collapsing a multi-step tool chain into a single LLM inference turn. This is the mechanics lesson complementing the prior execute_code vs terminal decision-rule lesson (2026-07-08-execute-code-vs-terminal-decision.md). Goal: give Alfred a precise mental model of the socket transport, the token-collapsing property, the available in-script tools, the terminal() bridge, and the code_execution.mode config knob. ...