Hypothesis Testing

Nested Ladder — Does Behaviour Beat History?

Primary test: M0 (Case) → M1 (+ Identity) → M2 (+ History) → M3 (+ Decision-Time Behaviour). Anthrocentrix is supported only when M3 outperforms M2 out-of-sample and survives leakage, permutation, and stability tests.

Benchmark Dataset Framework

cross-domain registry · pure-behaviour signal outside chess

Goal: determine whether the M3 − M2 evidence score replicates outside chess. Lichess is the reference benchmark; ingestion adapters for Immigration Judges and Federal Criminal Sentencing are registered with canonical field mappings and remain unwired until raw data is loaded. Numbers update automatically when a Purity Test runs against a registered dataset.

Dataset	Actors	Events	M2 Acc	M3 Acc	Evidence	Leakage	Stability	Status
Lichess Open Database Chess (move-level)	—	—	—	—	—	PENDING	PENDING	live
U.S. Immigration Judges (EOIR) Asylum adjudication	—	—	—	—	—	PENDING	PENDING	adapter ready
U.S. Federal Criminal Sentencing Judges Criminal sentencing	—	—	—	—	—	PENDING	PENDING	planned

Note. No new behavioural features are introduced by this framework. The four pure features (inter_sec, workload_24h, hour_of_day, seq_pos) carry over unchanged. A row is only marked CLEAN / STABLE after a Purity Test has actually run against the registered dataset.

EOIR Dataset Acquisition Wizard — Immigration Judges Benchmark

source → template → upload → map → validate → import (no code required)

1 · Source→

2 · Template→

3 · Upload→

4 · Map→

5 · Validate→

6 · Import

Fastest Path to >100,000 Decisions

EOIR Immigration Court Database (Hugging Face)

Ian Nason / Backlog Immigration LLC · DuckDB (.duckdb) — query then export to CSV

compat score

volume

25/25

ease

12/25

schema

20/25

truth

25/25

Estimated decisions: 7,000,000 · Time: 20–40 minutes · Ground truth: Available (BIA reversals)

Steps: Download the DuckDB (.duckdb) — query then export to CSV from https://huggingface.co/datasets/Nason/eoir-database/resolve/main/eoir.duckdb → Filter to merits decisions (exclude scheduling and admin closures) → Export to CSV with canonical column headers → Upload CSV to Fusion EOIR Acquisition Wizard → Map any remaining columns and run validation

Open download page

Pick a public EOIR-compatible data source. The wizard never fabricates data — every row must come from a real release. Sources are ranked by fastest path to a Fusion-compatible immigration-judge dataset with >100,000 decisions. Each card shows download URL, file format, estimated row count, required preprocessing, and a Fusion compatibility score (0–100).

EOIR Immigration Court Database (Hugging Face)

82/100GOOD·CLI required

https://huggingface.co/datasets/Nason/eoir-database/resolve/main/eoir.duckdb

The most comprehensive public EOIR dataset available. 164M+ rows across 97 tables built from monthly DOJ FOIA dumps. Includes cases, proceedings, hearings, charges, relief applications, and appeals. A single DuckDB file that can be queried with SQL before export.

Format

DuckDB (.duckdb) — query then export to CSV

Est. decisions

7,000,000

Required preprocessing

Install DuckDB CLI or Python duckdb
Query the cases/proceedings table: filter DEC_TYPE to merits decisions
Export filtered results to CSV (COPY ... TO 'eoir.csv' (HEADER, DELIMITER ','))
Rename extracted columns to canonical Fusion fields (ij_code, case_id, decision_code, decision_date)
Optional: join appeals table for appellate_outcome ground-truth proxy

• Best row-per-decision yield of any public source.

• DuckDB handles the multi-table join complexity for you — no manual CSV stitching.

• Ground-truth (BIA reversals) available in appeals sub-tables.

• Recommended for researchers comfortable with SQL.

DeportationData.org — Processed EOIR Cases

91/100EXCELLENT·Few clicks

https://deportationdata.org/data/processed/eoir.html

Academically processed version of the DOJ EOIR FOIA dump. Already cleaned, deduplicated, and formatted as CSV, Stata, SPSS, or Feather. Includes an interactive explore tool. The processing documentation explains every transformation applied.

Format

CSV, Stata (.dta), SPSS (.sav), Feather

Est. decisions

4,000,000

Required preprocessing

Download the CSV bundle from the processed data page
Filter to merits decisions (exclude scheduling, admin closures)
Rename columns to canonical Fusion fields
Convert Stata/SPSS categorical labels to string values if using those formats

• Fastest path for non-technical users — download a single CSV bundle.

• Already handles the notorious EOIR ZIP extraction bugs.

• Ground-truth available if appeals table is included in the bundle.

• Academic-quality documentation and codebook.

DOJ EOIR FOIA Library — Raw Monthly Dumps

65/100FAIR·Coding required

https://www.justice.gov/eoir/foia-library-0

The canonical government source. Monthly ZIP releases of case-level master tables (A_TblCase, B_TblProceeding, C_TblHearing, etc.) covering all immigration court proceedings since the 1970s. Updated monthly due to an original TRAC FOIA request.

Format

ZIP archive containing multiple CSVs

Est. decisions

3,000,000

Required preprocessing

Download latest ZIP from FOIA Library (may require handling extraction bugs — use 7-zip, Bandizip, or `unzip` CLI)
Join B_TblProceeding to A_TblCase on IDNCASE
Filter to merits decisions (DEC_TYPE in merits codes)
Map DEC_CODE to canonical grant/deny/other vocabulary
Parse COMP_DATE to ISO-8601
Optionally join appeals table for appellate_outcome

• Most authoritative source — straight from the agency.

• Requires multi-table join and column mapping.

• ZIP extraction is known to fail with standard archive tools; use CLI `unzip`.

• Updated monthly — good for longitudinal tracking.

TRAC Immigration — Asylum Decisions Tool

78/100GOOD·Few clicks

https://trac.syr.edu/immigration/tools/

FOIA-derived asylum decision summaries with stable judge identifiers. The canonical source for judge-level immigration research. Data is available through an interactive web tool; bulk download may require TRAC registration or institutional access.

Format

Interactive web tool + bulk download (registration may be required)

Est. decisions

1,000,000

Required preprocessing

Access TRAC asylum decisions tool and export data
Filter to asylum case-type rows
Rename TRAC fields to canonical Fusion fields (IJ → ij_code, IDNCASE → case_id)
Map DEC_CODE values (G/D/O) → grant/deny/other
Combine COMP_DATE (+ optional COMP_TIME) → ISO-8601 decision_date

• Gold-standard for judge-level asylum research.

• Bulk download access may be limited; check TRAC subscription requirements.

• Focused on asylum only — broader immigration proceedings not included.

• Appellate outcomes (BIA reversals) not consistently available.

OpenImmigration.us — Free JSON Datasets

0/100NONE·One click

https://www.openimmigration.us/downloads

Community-processed JSON datasets derived from EOIR data. Includes judge index (1,269 judges), court index, nationality index, and yearly trends. All data is aggregated statistics — NOT case-level.

Format

JSON (aggregated statistics)

Est. decisions

N/A (aggregate)

Required preprocessing

Download JSON files via curl or browser
NOT COMPATIBLE with Fusion — data is pre-aggregated, not one-row-per-decision
Can be used for cross-reference (judge names, court metadata) but not for behavioral signal extraction

• Fastest download — pure JSON, no registration.

• Completely incompatible with Fusion ingestion adapter (requires row-level decisions).

• Useful only as a metadata crosswalk (judge names ↔ codes, court locations).

• No ground truth, no timestamps, no individual decisions.

Data.gov — EOIR Catalog

15/100POOR·Few clicks

https://catalog.data.gov/dataset?q=immigration+court

Federal open-data catalog listing EOIR datasets. Mixed granularity — aggregate backlog tools, court operational status maps, and pointers to EOIR/TRAC releases. No direct bulk case-level download.

Format

Mixed (HTML, CSV aggregates, API pointers)

Est. decisions

N/A (aggregate)

Required preprocessing

Browse catalog to discover newer EOIR drops
Most datasets are aggregate — confirm row-level before downloading
If row-level found, apply same preprocessing as DOJ EOIR FOIA source

• Useful as a discovery layer for new EOIR releases.

• Most listed datasets are aggregate (backlog, court maps) — not case-level.

• Occasionally links to bulk CSVs that redirect to the FOIA Library.

• Not a primary acquisition path.

Run Kill Test

select an imported dataset · ≥10 decisions per actor required

No imported datasets available. Upload a CSV with actor_id, decision, ground_truth, timestamp from Datasets to enable the kill test. Seeded demo datasets are illustrative and do not carry the per-event records required for chronological splits.

Methodology

split Per-actor chronological 50/50. Training = first half of each actor's decisions. Validation = second half. Actors with fewer than 10 decisions are excluded.

baseline P(correct | decision) with Laplace smoothing. Falls back to global training rate when the validation decision was unseen in training. No actor information.

actor model Empirical-Bayes blend of P(correct | actor, decision), the actor's overall rate, and the case-only baseline. Weight on the joint estimate grows with its support.

verdict Absolute accuracy lift on the validation set: <1% NO SIGNAL · 1–5% WEAK SIGNAL · >5% STRONG SIGNAL.