Frontier Open VS Closed

Answer guidance

The classification stack is a hybrid rules-plus-ML pipeline built around honest uncertainty: ingestion lands all LMS/course items with metadata and extracts PDF text with OCR fallback; signal extraction then enriches each item with entity matches, OpenAlex/OpenAIRE DOI verification (license, OA status, title match), a separate document-type classifier (one of ~8 document types, kept out of the final label space), DINOv2 page-image embeddings, and sentence-embedding text-PCA dims, all materialized into a ~75-column features hub. The model itself is a deterministic focal LightGBM multiclass classifier, isotonic-calibrated, trained on manual reviewer labels (weighted by label provenance, duplicate-grouped) over a leak-free 88-feature matrix that deliberately excludes doctype opinions so the model learns independent evidence. Trust is gated by Mondrian (per-class) conformal prediction at 90% coverage. At read time a fixed rule chain composes the displayed suggestion — verified-DOI license/OA override first, doctype gate next, then the model, then honest "none" — over the target verdict categories, with human verdicts always overriding and never being overwritten. The human loop closes it: conformal set size and active-learning ranking drive the labeling queue, reviewer verdicts fan out to duplicates in-transaction, and every retrain folds them back in. Verification runs through duplicate-grouped CV, out-of-fold evals, conformal coverage checks, and an invariant-checking CLI; any weak edges identified are handled by surfacing model doubt into review rather than hiding it, which is the design's core stance: predict aggressively, but quantify exactly when you're guessing.