Skip to content

features

Feature extraction: LLM tagging pass + per-key click-preference scores.

Reliability contract mirrors explain.py: tagging is lazy relative to ingest, validated, and skips (never blocks) on failure. Nothing in this module runs on the rank path except pure-SQL/numpy preference scoring.

ItemTags

Bases: BaseModel

The tagger's structured reply: content type plus one to four tags, validated and normalized by _tags_shape rather than rejected whole.

TagPrompt dataclass

A tagging prompt as data: an instruction and optional demonstrations.

Produced offline by the optimizer (evals/optimize_tagging.py) and loaded here; the library never optimizes. source names the file it came from so a tagging run can report which prompt produced its tags, the way it reports which model did.

load_tag_prompt(path)

Read a prompt artifact, validating the parts the renderer relies on.

A demo is rendered verbatim as an assistant turn, so a malformed one -- a content_type the schema rejects, a tag the validator would strip -- would teach the model the exact output the run then discards. Refuse it here, before any model call.

pref_scores_for_items(conn, user_id, item_ids)

Mean per-key preference score per item, aligned with item_ids. 0.5 = neutral.

run_reference_tagging(conn, chat_fn, model, limit=None)

LLM-tag references with no tags yet, through the ONE tagging renderer.

Same contract as run_tagging: chat_fn and model come from the caller, a failed record stays untagged for the next run, and a dead backend stops the run with chat_down set rather than failing every row. The vocabulary is the corpus's (tag_vocabulary), so a reference is steered toward the tags the reader's items already use -- which is what lets it join the concept graph beside them.

run_tagging(conn, chat_fn, model, limit=None)

Tag every untagged item, newest first. Returns {"tagged": n, "failed": n}.

Failed items are skipped (stay untagged) and retried on the next run. chat_fn and model are the caller's concrete backend and its resolved name, required rather than defaulted -- this module reaches the model only through them, never through attestation.llm directly, so a second provider needs no change here. A silent domain-level default was tried and rejected: it fell back to the literal string "unresolved" as a model name with nothing exercising that path, which is a worse failure mode than the TypeError a caller who forgot to resolve model now gets.

tag_messages(title, summary, vocab, prompt=None)

The ONE renderer of the tagging prompt.

The eval harness, the optimizer's transfer test and attest tag all call this, so a score is always a score of the prompt that actually runs. With prompt=None the output is the hand-written prompt, byte for byte. Demonstrations render as prior user/assistant turns without the vocabulary line: 150 tags repeated per demo would spend more prompt on the examples than on the item.

tag_one_item(conn, item, chat_fn, vocab, model_name, prompt=None)

One LLM call (plus one retry) -> item_features + item_tags rows. False = skipped.

tag_prompt_from_env()

attest tag uses the artifact ATTEST_TAG_PROMPT names, else the hand-written prompt. Artifacts come from evals/optimize_tagging.py.

tag_vocabulary(conn, limit=150)

Most-used tags, canonicalized, to steer the model toward existing vocabulary.

Counts are summed over kg.canonical before ranking, which fixes a feedback loop: the raw table ranks each spelling separately, so the vocabulary shown to the model listed machine-learning (872) AND machinelearning (463), and llm (642) beside language-models (212), spending three of forty slots re-teaching the model the very variants the graph then merges away. Worse, the spellings it taught were the ones canonical() rewrites, so the model was being steered toward deprecated forms. Merging first frees those slots and lets real concepts (hugging-face, natural-language-processing, continual-learning) into the list instead.

limit is 150 rather than 40 because 40 covered only 59% of tag assignments on the live corpus against 67% at 150 -- a model shown 40 tags for a 5000-item archive meets an unfamiliar subject constantly and mints a new tag when it does. 150 canonical tags is ~1.4KB of prompt, which is affordable next to the item summary it accompanies.

Excludes NON_TOPIC_TAGS: the prompt asks the model not to emit them, so suggesting them here would work against it.

top_and_bottom_keys(conn, user_id, n=5)

The n feature keys this reader has scored highest and lowest, by the same Laplace-smoothed score the preference term ranks with. Public so the profile-status tool stops importing _key_stats/_score across the module boundary; the smoothing is not reimplemented. A key counts as liked only above 0.5 (neutral) and disliked only below it, so a reader with fewer than n clicks on either side gets a shorter list rather than a padded one.