We tested an MLB model against the closing line. Then we borrowed an idea, and the edge got bigger.

What We Built

A model that estimates the chance the home team wins

We pulled 4,857 regular-season games from 2024 and 2025 using the MLB Stats API and attached team-strength features to the matchups: prior-season Wins Above Replacement split into offense and defense, season-to-date win percentage going into the game, the ballpark run factor from Baseball Savant, and both teams’ preseason Steamer WAR projections. The features use information that existed before first pitch, so the model cannot peek at the result it is trying to predict.

We added the Steamer projection after testing it on its own. The preseason WAR figure freezes before opening day and joins a game by season alone, so it leaks no outcome information. The difference between the two teams’ projected WAR turned out to be one of the more useful inputs, which suggests the projection systems capture strength the in-season record has not caught up to yet.

The model is a logistic regression, chosen on purpose. A small model with a handful of plain features is easier to audit and harder to fool yourself with than a large one. The input with the largest weight is the home team’s prior-season WAR, which matches what you would expect.

Top features by weight

Weights come from the trained model on standardized inputs, using regressed versions of the rolling stats. The offense and defense splits trade off against the combined WAR terms, so individual signs read less cleanly than the totals.

How We Checked It

Train on the past, test on the future

We sorted the games by date, trained on the first 70 percent, and tested on the last 30 percent. The model learns from games that finished before the ones it gets graded on, the same position a real forecaster is in.

• Training set: the first 3,399 games in date order.
• Test set: the last 1,458 games, June 8 to September 28, 2025.
• The model picks the winner in 56.1 percent of test games.
• A baseline that picks the team with the better record gets 54.5 percent.

So the model adds 1.6 points over the obvious baseline. Picking winners and beating a betting market are different jobs, though. The market price folds in what our features capture plus injury news, lineups, and late money. To learn whether the model knows something the market does not, we compared it to the actual price.

The Real Test

Against the closing line

The closing line is the betting price just before first pitch. By the time it closes, the price has absorbed injury news, lineup cards, weather, and late money from well-informed bettors. A real edge shows up against the close or it does not exist.

We bought historical odds snapshots taken twelve minutes before first pitch for 354 test games, removed the bookmaker’s margin to recover the market’s implied win probability, and scored both forecasts side by side.

354 games with real closing prices from the held-out test window. The market leads by 1.4 points on winners, 0.002 on Brier score, and 0.005 on log loss. The model still does not out-predict the market on raw accuracy.

The market picks winners better and prices probabilities more accurately. That part has not changed. But accuracy is not the only way to beat a market. A model can be wrong more often and still make money if it is right about which games it is wrong about. That is where the next part comes in.

What We Borrowed

A shrinkage trick from an open-source baseball project

While evaluating a GitHub repository called RPCBaseball, we found a technique worth borrowing. The project, built by proselotis and msc123123, blends simulated win probabilities with observed frequencies using a simple Bayesian formula:

final estimate = (model estimate × sample size + prior × weight) ÷ (sample size + weight)

The idea is straightforward. Early in the season, you have very few games to learn from, so the model’s predictions are noisy. Instead of trusting them fully, you blend each prediction with a league-average prior. As the season goes on and more games pile up, the model earns more trust and the prior fades into the background.

We applied this in two places. First, we regressed the rolling team and pitcher stats toward the league average before feeding them to the model, so a team that is 4–0 does not look like a juggernaut. Second, we applied the same shrinkage to the model’s final output: early-season predictions get pulled toward the league-average home win rate of about 53 percent, and late-season predictions are left alone.

The Result

The edge got bigger

Before the shrinkage, betting only on games where the model disagreed with the market by at least five percentage points returned about 10 percent on flat stakes. After the shrinkage, the same threshold returned about 14 percent. The technique did not make the model smarter — its accuracy and Brier score barely moved. What it did was make the model less overconfident, so it stopped betting on marginal games where its edge was illusory.

354 games, flat $1 stakes, best available closing decimal prices. “Edge” is the gap between the model’s probability and the market’s implied probability. Positive ROI means the model beat the close.

The clearest gain came from the prediction-level shrinkage alone: +14.3 percent at the five-percent edge, up from +9.9 percent. The combined version — regressed features plus prediction shrinkage — is the most consistent, showing positive returns at every threshold we tested.

The Tempting Part

Why we still do not sell picks

The numbers above look better than they did before, and the improvement came from a principled technique, not from tweaking thresholds until the backtest looked good. We are still not selling picks, for the same reasons as before:

• The model still loses to the market on raw accuracy and calibration. The shrinkage improved the value-betting edge, not the probability quality.
• The sample is thin. Even after filtering, we are looking at a few hundred bets across one partial season. That is not enough to separate a real edge from a favorable run.
• The ROI concentrates at one threshold. The +14.3 percent figure lives at the five-percent edge; at other thresholds the combined model does better, the shrinkage-only model does not. That kind of sensitivity to a single parameter is a yellow flag.

We treat a backtest that needs the right slice to look good as a warning, even when the improvement comes from a sound idea borrowed from people who know baseball.