Promote Sign in Sign up

Repository

jlens-mood

A mood ring for text — emotion-axis readout from LLM residual streams via the Jacobian lens

eric-tramel
Repository

README

jlens-mood 🌈💍

A mood ring for text. Slip a piece of writing onto a language model's
finger — one prefill pass, no decoding, no prompting — and watch the
emotional weather swirl through its residual streams as it reads, via
Anthropic's Jacobian lens.
The model never says a word; the ring changes color anyway.

jlens-mood --tui reading the Tragedy of Darth Plagueis the Wise

jlens-mood --tui reading the Tragedy of Darth Plagueis the Wise: the ring
lands on fear and anger, "I thought not." reads as curiosity 45%, and the
river tracks the legend's mood token by token. It's not a story the Jedi
would tell you — but the residual stream will.

$ jlens-mood "The morning was perfect: sunshine, fresh bread, and laughter in
              the garden. Then the phone rang. The accident on the highway was
              fatal, and everything we had planned collapsed into grief."

mood: sadness (28%, intensity 3.36)

  sadness   ███████░░░░░░░░░░░░░░░░░  28%
  surprise  ████░░░░░░░░░░░░░░░░░░░░  18%
  fear      ████░░░░░░░░░░░░░░░░░░░░  18%
  joy       ████░░░░░░░░░░░░░░░░░░░░  17%
  ...

        joy  70%  The morning was perfect: sunshine, fresh bread, and laugh...
   surprise  37%  Then the phone rang.
       fear  38%  The accident on the highway was fatal, and everything we ...

Because the readout comes from internal states rather than surface vocabulary,
it captures things a lexicon can't: "Then the phone rang." after an idyllic
scene reads as surprise before any bad news is stated.

How it works

The Jacobian lens (J-lens) is a
per-layer linear map J_l = E[∂h_final,t' / ∂h_l,t] — the average effect of a
residual-stream state on the model's future outputs. Applying it to the
activations at any token position yields logits over the vocabulary describing
what that state is poised to say, now or later.

jlens-mood builds eight axes on top of that readout — six Ekman emotions
(joy, sadness, anger, fear, disgust, surprise), curiosity, and an anchored
neutral axis. Each axis is a cluster of anchor words verified to be single
tokens in the model's vocabulary; per position, an axis scores the mean lens
logit over its cluster, and shares are a softmax across axes after subtracting
each axis's baseline on neutral reference text (cluster token base rates
differ, and without calibration everything reads fear/surprise).

Two mechanisms decide "no particular mood":

  • an intensity gate — if no axis clears the calibrated threshold, the
    verdict is neutral regardless of shares (absence of emotion);
  • the anchored neutral axis — boring/routine vocabulary in the workspace
    (boredom as content). Calibration zeroes this axis on merely-flat text by
    construction, so it cannot fire there; the gate covers that case. In
    practice the gate renders most neutral verdicts — read the neutral axis's
    share as a blandness indicator (it runs ~2× its usual level on narrated
    tedium) rather than expecting it to win outright.

Everything runs on a pre-fitted lens from
neuronpedia/jacobian-lens
over Qwen3.5-0.8B — no fitting, no GPU required (MPS or CPU is fine).

Usage

uv sync

uv run jlens-mood "Some text to appraise."      # bars + per-sentence moods
uv run jlens-mood -f essay.txt                  # file input
cat essay.txt | uv run jlens-mood --json        # stdin, machine-readable
uv run jlens-mood -f story.txt --tui            # emotion-colored tokens +
                                                #   terminal share chart
uv run jlens-mood -f story.txt --plot mood.png  # matplotlib mood river
uv run jlens-mood "..." --tokens                # per-token dominant emotion

uv run jlens-readout "..." --top-k 5            # raw workspace: top-k lens
                                                #   tokens per position/layer

First run downloads the pre-fitted lens (~46 MB) and, if absent from your
HuggingFace cache, the Qwen3.5-0.8B weights (~1.6 GB).

Method notes

  • Raw prefill, no chat template — deliberately. The lens was fit on bare
    wikitext prefill, and an A/B on this 8-axis readout (scripts/chat_ab.py)
    shows templating changes the text-span scores negligibly (pearson r
    0.90–0.98, per-axis mean shifts ≤ 0.4 logits) while the positions after the
    generation prompt fill with reply scaffolding ("Congratulations", markdown
    fragments) instead of comprehension. Raw prefill measures reading, not
    replying.
  • Uniform-mean aggregation. Emotion identity is carried distributively
    across positions: trailing windows miss mid-sentence content and
    max-pooling chases single-token spikes (scripts/agg_ab.py compares).
  • Anchors are tokenizer-specific. Words are vetted single-token against
    Qwen3.5-0.8B; on another model the runtime re-checks and warns about drops.
    (Fun fact: nearly the entire English disgust register — revolting,
    repulsive, sickening, stench, putrid — is multi-token in Qwen's BPE.)
  • Verification battery: scripts/mood_battery.py, 9/9; probe texts avoid
    their target axis's anchor words, so a hit means the state was inferred,
    not echoed.

Limitations

  • Shares are relative (softmax) — the intensity value is the "how much mood
    at all" signal; with eight axes, ~25–40% dominant share is a strong verdict.
  • It reads the model's appraisal, an uncalibrated proxy for a reader's.
    Sarcasm doesn't resolve at 0.8B ("Just perfect." reads positive).
  • The neutral gate threshold (2.0) is calibrated on a small battery; borderline
    text near intensity ~2 will be jumpy.
  • Long inputs process in disjoint 256-token windows; context doesn't carry
    across window boundaries.
  • Anchor lists are hand-picked English words. Validation against a labeled
    corpus (e.g. GoEmotions) is future work.

Credits

License

MIT