{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ed7bc501",
   "metadata": {},
   "source": [
    "# Senate candidate study audit\n",
    "\n",
    "Earlier-election-only comparisons updated October 19, 2026, with source cutoff October 18. No new candidate-family weights are enabled. This notebook inspects saved results; preparation and comparison scripts reproduce the fits before publication. Source limitations are part of the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "87670df8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:35.075300Z",
     "iopub.status.busy": "2026-10-19T07:36:35.074303Z",
     "iopub.status.idle": "2026-10-19T07:36:36.158602Z",
     "shell.execute_reply": "2026-10-19T07:36:36.157577Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"status\": \"PASS\",\n",
      "  \"checks\": 27\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "import json,sys\n",
    "ROOT=Path.cwd()\n",
    "if not (ROOT/'senate_model_config.json').exists(): ROOT=ROOT.parent\n",
    "sys.path.insert(0,str(ROOT))\n",
    "O=ROOT/'model/extensions/senate_candidates'\n",
    "load=lambda f: json.loads((O/f).read_text(encoding='utf-8-sig'))\n",
    "from validate_senate_candidate_study import main\n",
    "main()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "73dbc9b9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:36.164607Z",
     "iopub.status.busy": "2026-10-19T07:36:36.163596Z",
     "iopub.status.idle": "2026-10-19T07:36:36.182372Z",
     "shell.execute_reply": "2026-10-19T07:36:36.180834Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Career {'baseline': 4.61251, 'controls': 4.49772, 'elected_terms': 4.49598, 'prior_office': 4.50584, 'open_seat_prior_office': 4.49913, 'nonincumbent_prior_office': 4.50558, 'office_hierarchy': 4.49577, 'prior_electoral_performance': 4.50115, 'terms_and_office': 4.50451, 'terms_and_performance': 4.49954, 'office_and_performance': 4.51127, 'all_careers': 4.50905, 'careers_and_context': 4.51415}\n",
      "News {'baseline': 4.61251, 'controls': 4.49772, 'news_severity': 4.50382, 'strict_severity': 4.49416, 'severity_categories': 4.49835, 'recent_news': 4.49965, 'unpriced_news': 4.4974, 'news_and_careers': 4.51531, 'strict_and_careers': 4.50868, 'all_candidate_context': 4.50824}\n",
      "Context {'baseline': 4.61251, 'controls': 4.49772, 'smaller_states': 4.48898, 'congress_approval': 4.49796, 'both': 4.48959}\n"
     ]
    }
   ],
   "source": [
    "career=load('career_comparison_analysis.json')\n",
    "news=load('scandal_comparison_analysis.json')\n",
    "context=load('incumbency_context_analysis.json')\n",
    "for family,d in [('Career',career),('News',news),('Context',context)]:\n",
    "    print(family, {k:round(v['MAE_pp'],5) for k,v in d['aggregate'].items()})\n",
    "    assert all(not g['versus_controls']['enabled'] for g in d['gates'].values())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d6f2ffe3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:36.186381Z",
     "iopub.status.busy": "2026-10-19T07:36:36.185375Z",
     "iopub.status.idle": "2026-10-19T07:36:36.196583Z",
     "shell.execute_reply": "2026-10-19T07:36:36.194561Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matched financial races: {'quarter': {'2016': 5, '2018': 11, '2020': 0, '2022': 13, '2024': 0, '2026': 28}, 'cumulative': {'2016': 5, '2018': 10, '2020': 0, '2022': 13, '2024': 0, '2026': 28}}\n",
      "quarter share_difference 3.6955 3.7783\n",
      "quarter absolute_individual_gap_millions 3.6955 3.4885\n",
      "quarter log_individual_ratio 3.6955 3.7967\n",
      "cumulative share_difference 3.6955 3.7436\n",
      "cumulative absolute_individual_gap_millions 3.6955 3.7084\n",
      "cumulative log_individual_ratio 3.6955 3.6921\n",
      "2020/2024 historical report gaps prevent a general financial-weight conclusion.\n"
     ]
    }
   ],
   "source": [
    "financial=load('individual_dollar_comparison_analysis.json')\n",
    "print('Matched financial races:',financial['coverage'])\n",
    "for r in financial['tests']:\n",
    "    if r['test_cycle']==2022: print(r['period'],r['feature'],round(r['baseline_MAE_pp'],4),round(r['candidate_MAE_pp'],4))\n",
    "assert not financial['enabled']\n",
    "print('2020/2024 historical report gaps prevent a general financial-weight conclusion.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d21ca2eb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:36.201126Z",
     "iopub.status.busy": "2026-10-19T07:36:36.201126Z",
     "iopub.status.idle": "2026-10-19T07:36:36.211660Z",
     "shell.execute_reply": "2026-10-19T07:36:36.210128Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Career source coverage: {'candidate_race_instances': 507, 'biography_instances': 452, 'current_candidates': 71, 'current_biography_instances': 62, 'review_flags': 57, 'incumbency_statuses': {'not a sitting senator': 262, 'elected multiple times': 97, 'elected once': 82, 'appointed; no prior Senate election win': 11, 'career research incomplete': 55}, 'status': 'Source extraction and chronological comparison complete with documented coverage gaps and office-parsing review flags.'}\n",
      "Event support: {'2010': {'races': 17, 'races_with_accepted_news': 3, 'races_with_major_news': 0}, '2012': {'races': 23, 'races_with_accepted_news': 2, 'races_with_major_news': 0}, '2014': {'races': 20, 'races_with_accepted_news': 1, 'races_with_major_news': 0}, '2016': {'races': 25, 'races_with_accepted_news': 2, 'races_with_major_news': 0}, '2018': {'races': 26, 'races_with_accepted_news': 2, 'races_with_major_news': 1}, '2020': {'races': 26, 'races_with_accepted_news': 6, 'races_with_major_news': 0}, '2022': {'races': 26, 'races_with_accepted_news': 3, 'races_with_major_news': 0}, '2024': {'races': 27, 'races_with_accepted_news': 5, 'races_with_major_news': 0}, '2026': {'races': 35, 'races_with_accepted_news': 11, 'races_with_major_news': 1}}\n",
      "Study validation: PASS\n",
      "Current evidence: C:\\Users\\bshp9\\OneDrive\\Documents\\ChatGPT\\Polling Model\\model\\extensions\\senate_candidates\\current_candidate_evidence.csv\n"
     ]
    }
   ],
   "source": [
    "print('Career source coverage:',load('career_preparation_summary.json'))\n",
    "print('Event support:',news['source_support'])\n",
    "print('Study validation:',load('validation.json')['status'])\n",
    "print('Current evidence:',O/'current_candidate_evidence.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8c522842",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:36.214668Z",
     "iopub.status.busy": "2026-10-19T07:36:36.214668Z",
     "iopub.status.idle": "2026-10-19T07:36:36.435773Z",
     "shell.execute_reply": "2026-10-19T07:36:36.434236Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16 fundamentals checks PASS\n",
      "lean_incumbency races 130 cycles [2016, 2018, 2020, 2022, 2024] polls 4.6125 nested 4.6116 fixed25 4.3812\n",
      "candidate_only races 130 cycles [2016, 2018, 2020, 2022, 2024] polls 4.6125 nested 4.6125 fixed25 4.9458\n",
      "candidate_lean races 130 cycles [2016, 2018, 2020, 2022, 2024] polls 4.6125 nested 4.6381 fixed25 4.3484\n",
      "candidate_lean_strict races 130 cycles [2016, 2018, 2020, 2022, 2024] polls 4.6125 nested 4.6388 fixed25 4.352\n",
      "lean_incumbency_generic races 79 cycles [2020, 2022, 2024] polls 4.7255 nested 4.8173 fixed25 4.3296\n",
      "candidate_lean_generic races 79 cycles [2020, 2022, 2024] polls 4.7255 nested 4.8703 fixed25 4.2823\n",
      "finance_core races 13 cycles [2022] polls 3.6955 nested 3.6955 fixed25 4.245\n",
      "candidate_money races 13 cycles [2022] polls 3.6955 nested 3.6955 fixed25 4.2247\n",
      "candidate_money_generic races 13 cycles [2022] polls 3.6955 nested 3.6955 fixed25 4.2225\n",
      "candidate_money_amount races 13 cycles [2022] polls 3.6955 nested 3.6955 fixed25 4.2239\n",
      "A fixed blend improves average error, but its gain is inconsistent across national cycles; the primary nested blend does not improve it.\n"
     ]
    }
   ],
   "source": [
    "from validate_senate_fundamentals import main as validate_fundamentals\n",
    "validate_fundamentals()\n",
    "fundamentals=load('fundamentals_index_analysis.json')\n",
    "for name,r in fundamentals['primary'].items():\n",
    "    print(name, 'races',r['test_races'],'cycles',r['test_cycles'], 'polls',round(r['nested']['baseline']['MAE_pp'],4),'nested',round(r['nested']['candidate']['MAE_pp'],4),'fixed25',round(r['fixed25']['candidate']['MAE_pp'],4))\n",
    "assert not fundamentals['enabled']\n",
    "print('A fixed blend improves average error, but its gain is inconsistent across national cycles; the primary nested blend does not improve it.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2fb710e9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:36.439770Z",
     "iopub.status.busy": "2026-10-19T07:36:36.438775Z",
     "iopub.status.idle": "2026-10-19T07:36:37.032029Z",
     "shell.execute_reply": "2026-10-19T07:36:37.030007Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42 factor-removal checks PASS\n",
      "core 32 subsets 130 races full fixed25 4.348375 earlier-selected subset 4.641892\n",
      "Retrospective best groups: ['lean', 'incumbency', 'prior_performance', 'experience', 'scandals']\n",
      "generic 64 subsets 79 races full fixed25 4.282348 earlier-selected subset 4.8486\n",
      "Retrospective best groups: ['lean', 'incumbency', 'prior_performance', 'experience', 'generic_ballot']\n",
      "financial 128 subsets 13 races full fixed25 4.222457 earlier-selected subset 3.695465\n",
      "Retrospective best groups: ['lean', 'incumbency', 'prior_performance', 'generic_ballot']\n"
     ]
    }
   ],
   "source": [
    "from validate_senate_fundamentals_ablation import main as validate_removals\n",
    "validate_removals()\n",
    "removals=load('fundamentals_ablation_analysis.json')\n",
    "for name,r in removals['cohorts'].items():\n",
    "    print(name, r['subsets'],'subsets',r['test_races'],'races', 'full fixed25',round(r['full_fixed25']['candidate']['MAE_pp'],6),'earlier-selected subset',round(r['earlier_selected_subsets']['candidate']['MAE_pp'],6))\n",
    "    print('Retrospective best groups:',r['retrospective_best_fixed25']['groups'])\n",
    "assert not removals['enabled']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0f01581b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-19T07:36:37.038028Z",
     "iopub.status.busy": "2026-10-19T07:36:37.037032Z",
     "iopub.status.idle": "2026-10-19T07:36:38.337533Z",
     "shell.execute_reply": "2026-10-19T07:36:38.336512Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28 expert-rating checks PASS\n",
      "final 130 races polls 4.612505 with expert fundamentals 4.285707\n",
      "cutoff 120 races polls 4.44105 with expert fundamentals 4.427565\n",
      "final_on_cutoff_scope 120 races polls 4.44105 with expert fundamentals 4.357496\n",
      "Final ratings can use later information than the polling cutoff; the archived-cutoff comparison does not validate a new weight.\n"
     ]
    }
   ],
   "source": [
    "from validate_senate_expert_ratings import main as validate_experts\n",
    "validate_experts()\n",
    "experts=load('expert_ratings_analysis.json')\n",
    "for mode,r in experts['results'].items():\n",
    "    q=r['indices']['experts_and_candidates']['nested']\n",
    "    print(mode, r['indices']['experts_and_candidates']['test_races'],'races','polls',round(q['baseline']['MAE_pp'],6),'with expert fundamentals',round(q['candidate']['MAE_pp'],6))\n",
    "assert not experts['enabled']\n",
    "print('Final ratings can use later information than the polling cutoff; the archived-cutoff comparison does not validate a new weight.')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b782f5",
   "metadata": {},
   "source": [
    "The residual candidate additions and the primary learned fundamentals blend do not validate a new weight. A fixed 25% blend is more promising, but the gain is mainly available from partisan lean and incumbency, varies across elections and is not statistically reliable in the whole-election sensitivity test. The full five-family core score is also the best of all 32 retrospective fixed-blend subsets; earlier-selected subset search does not improve the current forecast. Expert final-rating benchmarks are promising, while archived-cutoff ratings add little; final-week information is not treated as known sixteen days before the election. This does not mean money, reputation, experience or public misconduct never matter. Development-cycle reuse, missing biographies and incomplete financial history limit inference. Forecast numbers and House outputs remain preserved; Senate descriptions and downloads record the research."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Polling analysis",
   "language": "python",
   "name": "polling-analysis"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}