crowmaki/hermes-learning
Watch
1
0
Fork
You've already forked hermes-learning
 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

feat/pedagogy-plugin #4

Merged
crowmaki merged 160 commits from feat/pedagogy-plugin into main 2026-08-29 02:21:05 +00:00
Conversation 0 Commits 160 Files changed 95 +27882 -69
crowmaki commented 2026-08-29 02:20:58 +00:00
Owner
Copy link
No description provided. 
Installing into ~/.hermes/hermes-agent/venv is unsafe: hermes update can
recreate it (Docker, desktop binary). Follow the mnemosyne convention of
plugin-owned environments; jsonschema (the only runtime dep) already ships
with Hermes, so install.py --check only reads that venv.
Draft 2020-12 schemas for RequestInterpretation, DiagnosticQuestion,
SufficiencyDecision, Syllabus, ModuleSourceSet, TutorTurn, Assignment,
DegradedKnowledgeReport, ChallengeResponse, plus CourseDocument, Settings
and the event envelope with per-type payload schemas.

Question choices require exactly one __dont_know__ sentinel (text
"I don't know") via contains/minContains/maxContains and >=2 other
choices; check_schema post-checks correct_choice_id against the choices
and rejects the sentinel as a key. Raises SchemaValidationError with
JSON-path error messages.
Valid/invalid fixtures for every lane, sentinel exactly-once, minimum
substantive choices, missing-key rejection, strict persisted docs, event
envelope/payload dispatch.
- CourseDocument gains interpretation and assignments (by id)
- DiagnosticStateDoc / FinalExamStateDoc defined explicitly
- attempt_started carries module_id; attempt_submitted uses unanswered_question_ids
- TutorTurn claims carry ids so challenges can reference them
- question post-check runs only after schema validation passes and
  considers str ids only (Counter), so non-hashable ids can't raise
- validator cache holds named SCHEMAS entries only; ad-hoc dicts uncached
- CourseDocument snapshot fields from the events fold: interpretation,
  assignments, ModuleState.passed_attempt_id, FinalExam.completed_attempt_id,
  Attempt.unanswered_question_ids, nullable learning_request and
  Diagnostic.closure
- event payloads: attempt_started.module_id required, optional
  objective_evidence.evidence_id, ModuleSourceSet.module_id required
- nullable via anyOf with deepest-branch error rendering (precise paths)
- objective_ids required on questions; explanation/choice.text non-empty
- TutorTurn.claims[].id required; is_structured_result_valid needs dict
An append after a truncated last line glued the new event onto the
fragment, so an acknowledged write was lost on the next load (REQ-015.1).
Also: a failed create_course no longer leaves a directory that blocks
retries; list_courses skips invalid directory names; module ids use the
same validator as course ids; expected_revision must be an int.
esbuild (esm, bundle, jsx automatic, externals limited to @hermes/plugin-sdk,
react, react/jsx-runtime) emitting the single dist/plugin.js, a post-build
lint that fails on foreign imports, surviving JSX, React.createElement or
hard-coded colours, tsconfig with jsx: react-jsx, and local type
declarations for the SDK subset in use (the SDK is not an npm package).
createApi(ctx) wraps ctx.rest: idempotency_key / expected_revision are
injected into the JSON body (query string on DELETE) because
PluginRestOptions has no headers; ApiError decodes the backend envelope
from the bridge's '<status>: <body>' rejection. useOperation polls
GET /operations/{id} every 1.5 s via the app's React Query client, stops on
succeeded|failed, maps kind -> pending label and surfaces Retry after 60 s
(POST /operations/{id}/retry). useMutationGuard disables controls while a
gesture is in flight and retries with the same key and input.
Default export { id: 'hermes-learning', name, register(ctx) } contributing
a ROUTES_AREA page at /learning and a SIDEBAR_NAV_AREA row. HealthGate
calls GET /health and renders the plugins.enabled + dashboard-restart
instructions when backend_registered is false (or the call fails); the
placeholder course list warns when no search provider is configured.
Hermes contributed routes are exact-match single-segment (apps/desktop
routes.ts), so course/module/assessment views are sub-views of /learning
selected via plugin-local state. Also documents the bridge's error
transport format.
ctx.rest rejections arrive as "Error invoking remote method 'hermes:api': Error: <status>: <body>";
the anchored status regex never matched, so every backend error decoded as 'unreachable'.
Pure PrerequisiteClosure (backward chaining from the target outcome,
frontier ordering, answer folding incl. the don't-know sentinel, local
sufficiency rule, foundation fallback) plus model calls for prerequisite
planning, next question (goal + compact summary payload) and sufficiency
confirmation via a private retrying _call. Adds the PrerequisitePlan
schema. step() orchestrates one diagnostic turn for the routes (4.2).
scripts/install.py (stdlib only): --check (node/npm, HERMES_HOME,
config.yaml, jsonschema importable from the Hermes venv interpreter,
read-only), npm build plus a Python port of lint-bundle.mjs, staged
assemble inside plugins/ with INSTALL_MANIFEST.json (version, git_sha,
built_at, sha256 per file), os.replace swap with .prev rollback,
post-install report (enable command, dashboard restart, desktop half
opt-in in Settings -> Plugins, reload desktop plugins), --enable via
`hermes plugins enable hermes-learning`, --uninstall (plugin dir only,
never plugin-data/), --dev symlinks + esbuild watch, --skip-build and
--hermes-home/--npm/--node/--hermes overrides for tests.
Tutor contract for module instruction: module-scoped teaching
(REQ-007.3), checks with the __dont_know__ sentinel and check-only
immediate feedback (REQ-007.2, REQ-002.2/6, REQ-010.9), cited claims
vs. reasoning spans and no invented citations (REQ-018.6/8), challenge
ladder identify -> confidence + rationale -> research -> retract/
qualify/unsupported (REQ-018.9-14), checkpoint discipline (REQ-007.6/7)
and the TutorTurn JSON shape from schemas.py.
Temp-HERMES_HOME install: expected tree (no tests/__pycache__), manifest
hashes verified, Python lint passes on the installed bundle, installed
tree imports in a subprocess the way both host paths do; reinstall
leaves plugin-data/ byte-identical; simulated os.replace failure
restores .prev and cleans the stage dir; uninstall removes only the
plugin dir; --check passes with a fake venv python and writes nothing,
fails when jsonschema is missing, and targets the venv interpreter
rather than sys.executable; lint rejects disallowed imports, raw JSX,
colours, extra files, require(); --enable shells out to hermes CLI
without editing config.yaml; --dev symlinks.
- Route every model call through agents/_common.structured_call (input as
  PluginLlmInput blocks; provider exceptions -> retryable generation_failed);
  resolve schemas/validators/api_errors via _common.plugin_module.
- next_question_id derives dq<n> from the max existing suffix, unique.
- Planning the target requires >=1 prerequisite (retry, then failure).
- closure.truncated set when depth/node bounds cut expansion; truncated or
  empty closures make offers low-confidence and empty closures foundation.
- Plan ids naming the target are skipped.
- step() seeds nodes from existing questions when the closure is missing.
- answered_count(state) is the single source for summary/no_competency/
  early_exit_allowed.
Directory plugins' skills never enter ~/.hermes/skills; the host only
exposes them through ctx.register_skill from register(ctx).
- diagnostic_question_added payload is now {question, closure}: the closure
  is persisted with every question because its expansion depends on model
  output and cannot be rebuilt deterministically across restarts.
- New events diagnostic_offer_made{reason, low_confidence, summary, closure}
  (pending syllabus offer, folded into diagnostic.offer/closure) and
  diagnostic_offer_declined{} (clears offer, bumps continue_count; REQ-003.10).
- DiagnosticState gains optional offer/continue_count (legacy docs validate);
  _new_diagnostic_state seeds them so the golden snapshot changed deliberately.
- diagnostics.step(force_question=) lets a declined offer yield one more
  probe past sufficiency/cap, deepening the closure when the frontier is empty.
GET /courses/{cid}/diagnostic (pure DiagnosticView with placement
disclosure), POST .../next (202 diagnostic_question operation, or 200 view
while a question is pending), .../answer (records diagnostic_answer, no
feedback, sentinel allowed), .../finish (diagnostic_too_short below 3,
else early_exit low-confidence offer; diagnostic stays open), .../continue
(diagnostic_offer_declined then one forced probe past sufficiency/cap).
diagnostic_closed is left to the syllabus publish route (task 5.1).
Covers disclosure, one-question-at-a-time, no correctness leaked on answer,
finish-at-2 rejected / finish-at-3 offer, CourseView syllabus_offer,
continue declining offers (early_exit, sufficient, cap_reached), cap
warning, idempotent replays, revision conflicts, generation failure, and
closure persistence across a fresh store.
- step(force_question=True) always yields a question: deepen the shallowest
  unexpanded node; if nothing can be deepened re-probe the node with the
  least evidence (dont_know < unsatisfied < satisfied); an empty closure
  gets a foundation-level node about the target (REQ-003.10).
- diagnostics.close_events(doc) builds diagnostic_closed{reason, closure}
  from the pending offer (validation_error when none); the syllabus publish
  route (5.1) appends it inside its transact — documented in the route module.
- POST /diagnostic/next returns 200 with the current view while an offer is
  pending (no op, no model call); /continue is the only way past an offer.
- POST /diagnostic/continue is refused while a diagnostic_question op for
  the course is in flight so a decline is never lost to an op replay.
Structured-lane agent: Syllabus schema + validate_syllabus gate inside
structured_call (3-try cap, defects fed back), compact diagnostic summary
as model input, normalised ModuleDoc shape in prerequisite-first order.
generation_failed now names the defects in its message so operation
records (which drop extra) still say what was wrong (REQ-004.7).
projections.ModuleLocked / require_module_enterable / enterable: import-free
403 module_locked carrying unmet_prerequisite_ids (REQ-006.5/6).
api_errors.coerce also picks up a foreign exception's 'extras' attribute.
POST /courses/{cid}/syllabus (operation: generate + validate + close
diagnostic + publish in one transact; diagnostic_open 409 without an offer
or while a question op is in flight), GET /courses/{cid} CourseView,
GET /courses/{cid}/modules/{mid} ModuleView (REQ-004, REQ-006, REQ-005.3).
A model-supplied id like 'a/b' or '..' could be published (the syllabus is
immutable, REQ-004.9) and then never reach /modules/{mid} or the transcript
file, stranding the module. Also assert storage extras reach the envelope.
checkpoint_saved{objectives_covered?} folds as a union into
modules[mid].objectives_covered (REQ-009.1 unlock rule input); SourceDoc
gains optional objective_ids (REQ-018.3). Golden snapshot updated.
build_source_set: <=2 web searches per objective (<=8 total) via
operations.run_blocking, async extract, one structured ModuleSourceSet
evaluation (authority/relevance/recency/corroboration, quotes,
objective_ids, disagreements; urls pinned to fetched candidates).
expand: challenge-driven additions with continuing s<n> ids. The host's
no-provider string is detected by prefix (fixture pinned) ->
search_provider_missing; missing extract provider degrades to snippets.
detect_challenge (ChallengeResponse) and turn (TutorTurn) with the
pedagogy-tutoring skill embedded as instructions; citations validated
against the module source set (uncited claims only under
challenge.action=unsupported), objectives/check ids checked, host-assigned
check ids.
enter (research gate / begin turn / checkpoint restore), say (tutor_turn op
with challenge -> expand -> answer), check-answer (sync local grading, sole
producer of objective_evidence, unlock rule), transcript?after= and sources
GETs; tutor checks redacted on the way out with check_id/answered.
remaining_objective_ids now empties once every objective is taught or
evidenced (REQ-009.2); needs_check flags that a correct instructional
check is still required for the unlock (REQ-009.1).
useOperation now exposes failed/retryable and reports retry_available for
a failed-retryable op (the backend accepts POST /operations/{id}/retry for
those); retry() resolves whether the backend accepted it. The module page
restores the sent text into the composer on failure, shows the envelope
message, and its Retry uses the op retry path, falling back to a fresh say
with the same text when that path is unavailable.
With active_attempt_id set the objectives panel (and the review notice)
navigate to the attempt sub-view; POST .../attempts only runs when no
attempt is active.
CheckCard honours the transcript's check.answered flag and the learner
check_answer line for the same question id (recordedAnswers), rendering
the recorded choice and correctness instead of an answerable form. System
transcript lines render as a centred note.
- check expected_revision under the course lock (transact) for start, answer and submit
- refuse answers/submissions on archived courses (course_locked), re-checked in the closure
- recompute homework/final policy inside the start transact; persisted kind and is_review follow it
- resume path returns 202 with an already-succeeded operation record (record_completed)
- idempotency key replay must match the subject (module id / final-exam), else validation_error
- move EXTRA_UNANSWERED_NUMBERS into api_errors next to the other EXTRA_* keys
- docstring: cross-course attempt ids are 404 attempt_not_found until task 8.1
- tests: stale revision 409s, archived course, key scoped to subject, unrounded 66.6 vs 67
A succeeded research enter-op posts enter again so the backend launches the
BEGIN tutor turn (REQ-007.2); the Begin button that sent a say is gone. The
mount-time enter waits for the course view to report the module enterable.
Transcript polling passes ?after=<last seq> and merges new lines. Stale
'task 6.2' comments updated; module_not_found added to the check-answer
404 whitelist.
- module passed between generation and commit persists kind=review / is_review=True
- with the lock-free pre-check disabled, stale expected_revision still 409s on answer, submit and start (op fails revision_conflict)
- unlock rule uses objectives_covered ∪ evidenced objectives in both the
  tutor save and check-answer (design §Homework unlock)
- 'unsupported' exempts only the claim named by challenge.claim_id; a
  challenged turn must carry challenge{claim_id, confidence, rationale}
  (REQ-018.8/9/10)
- expected_revision accepted on enter/say/check-answer (409 revision_conflict)
- check-answer transacts evidence first (re-checking 'answered' under the
  course lock), appends the answer line only afterwards; the line now
  carries the check's explanation for the UI (REQ-008.2)
- routes persist the learner/system line before creating the op, so retry
  and relaunch never duplicate it; enter gates BEGIN on 'no tutor line yet'
- say while a tutor turn is in flight -> 409 operation_in_flight {op_id}
- enter/check-answer idempotency replay matches kind, course and subject
- research: host-tools ImportError -> retryable generation_failed
- GET /transcript?after=<bad> returns the plugin error envelope
- tests for all of the above
- say requires the opening tutor turn (409 operation_in_flight while BEGIN
  runs, else 422 'enter the module first'); enter writes the system line
  whenever none exists
- superseded BEGIN op re-run is a no-op returning the existing opening turn
- check_answer.explanation is an explicit redaction exemption
  (projections.REDACTION_EXEMPT_CONTAINERS); redaction walk after an answer
- learner line carries idempotency_key: same-key retry after a failed start
  never re-appends; foreign op from the start race -> operation_in_flight
- say replay matches the module subject; EXTRA_OP_ID exported
Replace AttemptStubPage with AttemptPage: threshold banner before
question 1 (REQ-016.9), question stepper rendering from the server's
saved answers with each selection PUT immediately (REQ-010.3/4/7),
explicit "I don't know" sentinel choice (REQ-010.9, REQ-013.11),
submit confirmation listing unanswered question numbers and sending
their ids as acknowledge_unanswered with 409 recovery (REQ-010.5/6),
and a results view with score vs threshold, pass/fail, per-question
correctness + explanation and objectives to study (REQ-011.2-4). Review
sessions state that they never revoke a pass.

assessment.ts holds the AttemptView / ResultsView DTOs, query keys and
pure helpers mirroring plugin/dashboard/routes/attempts.py.
- agents/review.py: local, deterministic mapping of wrong answers (incl. the
  I-don't-know sentinel) on objectives of already-passed modules to
  review_suggested events (module_id + objective_id + evidence source
  check|homework:<aid>|final:<aid>), validated against DegradedKnowledgeReport;
  pending pairs are not duplicated
- attempts.py: one hook in the grade closure, same transact as attempt_graded
- instruction.py: wrong check on a passed module -> suggestion (REQ-012.6)
- routes/review.py: GET /courses/{cid}/review-suggestions and
  POST .../{sid}/dismiss (idempotency_key in body, idempotent, 404
  suggestion_not_found, course_locked when archived)
- projections.pending_suggestions made public; api_errors suggestion_not_found
- tests/test_api_review.py covers REQ-012.1/2/4/6/7/8 and REQ-013.1/2/9/10/11
- Never disable the answer radios while saving (aria-busy + status text
  instead) so arrow-key navigation keeps focus; coalesce selections with
  a short debounce and a single in-flight PUT, sending the latest choice
  once the current save resolves (no dropped selections).
- On a failed save render from the server's answers again, announce
  'Not saved.' and keep the inline error + retry (same key).
- Mint a fresh idempotency key per submit confirm click; after a 409
  unanswered_questions acknowledge the backend's own
  unanswered_question_ids rather than the page's possibly stale view.
- resultsKey is disjoint from attemptKey so invalidating the attempt no
  longer discards the seeded ResultsView.
- Nits: drop aria-label on <strong>, keep the focus outline on the
  question heading, correct-count wording for the empty study list.
- agents/review.py: skip per_question entries with chosen None (acknowledged
  skips); the I-don't-know sentinel still counts (REQ-012.6)
- events.py: review_dismissed for an unknown suggestion_id raises InvalidEvent
- tests: skip vs sentinel, cross-module ownership (route + unit, owner beats
  question.module_id), grade-transact atomicity, replay from log, fold guard
- routes/settings.py: GET/PUT /settings (mastery_threshold 0-100, default 75,
  invalid_threshold); threshold_changed recorded once in settings.json history
- routes/courses.py: POST /courses/{cid}/archive|restore (one event via
  transact, idempotent), DELETE /courses/{cid}?confirm=true&idempotency_key=..
  (confirmation_required, operation_in_flight, deletion_incomplete)
- storage: CourseStore.check_reference/find_owner shared cross-course guard,
  ModuleNotFound/AttemptNotFound/InvalidThreshold, set_mastery_threshold,
  DeletionIncomplete retryable
- operations: OperationRunner.inflight_for_course
- tests/test_api_settings.py contract tests (REQ-014, REQ-016, REQ-017, NFR-003)
- check_reference applied in attempts (_require_module/_find_attempt),
  instruction (_module_doc/_module_view), syllabus get_module, review _find
  (suggestion_id supported): other course's id -> 400 cross_course_reference
  with owner_course_id; unknown -> 404
- archived courses refuse POST /syllabus and diagnostic next/answer/finish/
  continue (409 course_locked, re-checked under the lock)
- archive refuses with operation_in_flight like DELETE
- DELETE runs under the per-course asyncio lock (delete_course_locked)
- corrupt settings.json is moved aside (settings.json.corrupt-<ts>), never
  overwritten
- tests updated/added in test_api_settings, test_api_syllabus,
  test_api_attempts, test_api_research
- useFocusHeading re-runs on every plugin-local nav transition (same-component
  view changes included), so focus lands on the new view's h1 consistently
- persistent polite LiveRegion per page/dialog for op progress, polling and
  load states (module enter/say/attempt, final exam, interpretation, list)
- status badges use StatusText (decorative glyph + sr-only prefix + text)
  instead of aria-label on generic spans
- course DAG: visible focus ring on the pan/zoom surface, arrow/Home/End
  navigation between nodes, zoom level exposed as a named status
- module popover: heading level h2, focus ring while the container is
  focused, sr-only Escape hint
- new-course dialog is explicitly labelled by its title
npm test bundles desktop/test/*.test.tsx with esbuild (SDK aliased to
test/sdk-mock.tsx) and runs them under node --test: CourseListPage and
CourseDiagramPage render in jsdom with fixtures; every control must expose an
accessible name, axe-core must report no violations, and the primary heading
must take focus after mount. Dev deps: react, react-dom, jsdom, axe-core.
docs/a11y-keyboard-walkthrough.md is the manual checklist for all four views.
- live regions carry progress only; errors are left to the role=alert
  InlineError so nothing is read twice; spinners beside a LiveRegion use the
  default label; course list no longer announces a count on every visit
- CourseDag pans the keyboard-focused node into view and settles wheel-zoom
  bursts before announcing the zoom level
- tests: ModulePage (transcript, check card, citation chip -> sources panel,
  composer, objectives) and AttemptPage (stepper, radios, save status, submit
  confirmation) spot checks; mocked api.post answers {}
- walkthrough: exact View results name, ConfirmDialog focus marked verify,
  updated live-region expectations; a11y.ts comment corrected
crowmaki deleted branch feat/pedagogy-plugin 2026-08-29 02:21:05 +00:00
crowmaki referenced this pull request from a commit 2026-08-29 02:21:07 +00:00
feat/pedagogy-plugin (#4)
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
crowmaki/hermes-learning!4
No description provided.