Distance results vary across model specifications
Preliminary finding.
Finding
Dropping the Portland facility location, which reassigns six county centers, leaves the distance term at about p 0.098, no longer conventionally significant. A 5,000-shuffle permutation test finds the distance coefficient exceeded in 3 of 5,000 shuffles (fraction 0.0006), but the shuffle randomizes the whole outcome rather than testing a null conditioned on the other covariate. A 50-mile threshold indicator, covering the 26 of 39 counties beyond that distance, is also associated with the rate but measures county-center geometry, not any postal route or pickup schedule.
- Retrospective specification
- A statistical fit over the 39 county rows of one past election's ballot records. Each joint fit carries two terms: distance in miles, and the county's recorded two-ticket share of the November 2024 presidential vote. The rate fits are ordinary least squares on Too Late ballots per 1,000, so a distance estimate is per 1,000 per mile. The count fits use a log link with the returns as an exposure offset and the quasi-Poisson dispersion convention, so a distance estimate there is a log-rate coefficient.
- County center beyond 50 miles
- True when the straight-line distance from the county's 2020 population-weighted mean center to the nearest of the three curated facility locations is more than 50 miles. It is geometry between two coordinates. It locates no post office, assigns no route and observes no pickup schedule.
Is the distance finding the same in every version of the model tested? No. Dropping one facility location leaves the distance term at about p 0.098, which is not conventionally significant, showing the result depends on how the model is specified.
Material qualifications
- The distance estimate stays positive in every specification tested, but it is not conventionally significant in all of them: dropping the Portland facility location, which reassigns six county centers, leaves the estimate at 0.187 per 1,000 mail ballots per mile with a conventional p of about 0.098.
- The 5,000-shuffle procedure shuffles the entire outcome across counties, so it tests a null of no association with either variable in the model, not a null conditioned on the other variable.
- The 50-mile indicator compares county population centers to curated coordinates; it does not locate a post office, assign a route, or observe a pickup schedule.
- The original project's verification script did not check the saved covariate column and did not run the leave-one-out, influence or facility-set checks it described; an independent audit reproduced those afterward.
Exact quantities
Scroll horizontally to see all columns.
| Quantity | Value | Unit | Denominator | Uncertainty |
|---|---|---|---|---|
| Distance term with the Portland location dropped | 0.187 | Too Late per 1,000 mail ballots per mile | The contrary specification: the distance term is not conventionally significant once the Portland location is dropped and six county centers are reassigned. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
| Conventional p-value for the distance term without the Portland location | 0.098 | probability | Thirty-six residual degrees of freedom. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
| Fitted difference for county centers beyond 50 miles | 16.6 | Too Late per 1,000 mail ballots | The county centers beyond 50 miles, against those within it. Proxy geometry, not policy exposure. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
| Conventional p-value for the 50-mile indicator | 0.024 | probability | Thirty-six residual degrees of freedom. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
| Shuffles at least as extreme as the observed distance term | 0.0006 | share of 5,000 shuffles | Three of five thousand shuffles of the whole outcome. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
| Shuffles at least as extreme as the observed share term | 0.2680 | share of 5,000 shuffles | The same procedure applied to the other term of the same fit. | Reproduced from the retained sources by an independent reviewer whose separate implementation agreed with the original run. |
How the evidence supports this finding
The same specification was refitted 39 times leaving out one county each time, once with the Portland facility location removed, once with a 50-mile indicator in place of the distance, and 5,000 times on a shuffled outcome.
Does the distance association hold up under different specifications?
Dropping one facility location leaves distance at about p 0.098; a permutation test and a 50-mile threshold give other results, so the association is specification-dependent.
The distance estimate stays positive in every specification tested, but it is not conventionally significant in all of them: dropping the Portland facility location, which reassigns six county centers, leaves the estimate at 0.187 per 1,000 mail ballots per mile with a conventional p of about 0.098.
The 5,000-shuffle procedure shuffles the entire outcome across counties, so it tests a null of no association with either variable in the model, not a null conditioned on the other variable.
Sources and calculations:
Evidence that could change the finding
A different facility set or a conditional permutation design would test whether the association is robust.
- Counterevidence considered
- Reproduced estimates for every published specification
Sources and calculations
Scroll horizontally to see all columns.
| Evidence | Locator | Source | Publisher |
|---|---|---|---|
| Reproduced estimates for every published specification | data selection: All 8 rows of retrospective-models.csv, plus the ranges recorded in the receipt: no specification that was run is excluded, including the one where the distance term is not conventionally significant. | Penner Strategy derived model input and result tables | Penner Strategy |
| The 39 county rows every specification is fitted on | data selection: All 39 rows of model-inputs.csv; no county is excluded, including the county that records zero Too Late ballots. | Penner Strategy derived model input and result tables | Penner Strategy |
The leave-one-out fits for all 39 counties, the influence diagnostic, the two-facility sensitivity, the 5,000-shuffle procedure and the 50-mile indicator were each reproduced independently. The original project's own verification script did not run several of them and did not check the covariate column, so its earlier exit status is not credited with this coverage.
Data, code and downloads
Every file this finding rests on. Figures carry their own download links beside them.
- Retrospective specifications and their estimates
Every specification this edition publishes, with each term's estimate, standard error, conventional p-value and the note that says what it measures.
- Retrospective specification receipt
What was reproduced in each specification, where the covariate comes from, and the list of things these numbers do not establish.
- Downloadable reproduction of the published specifications
Frozen public inputs, pinned dependencies, the calculation, a verifier, a permutation-schedule regeneration script and the expected value for every estimate this edition publishes. Its ballot-count input is coarsened to 480 nonpersonal county groups: no individual or precinct-level record is included.
Later changes
Updates to this finding. Corrections and withdrawals are listed with the current answer.
Use this research
Penner Strategy. "The distance association is not stable across every specification tested". How often Washington mail ballots are rejected for arriving late. Version 0001. September 12, 2026. https://pennerstrategy.com/research/washington-late-ballots/editions/0001/findings/f-model-sensitivity/. Qualification: The distance estimate stays positive in every specification tested, but it is not conventionally significant in all of them: dropping the Portland facility location, which reassigns six county centers, leaves the estimate at 0.187 per 1,000 mail ballots per mile with a conventional p of about 0.098.
- Print this edition
- Machine-readable release (JSON)
- Supporting files are available from the original source. See the evidence records for each source link.