← Blog

Predicting the next pitch, by decomposition

2026-06-23

A batter stepping in is not solving a random classification problem. The pitcher has a repertoire, a ball-strike count that shifts the odds, fatigue late in a start, a scouting report, and a memory of what was just thrown. That makes the next pitch partly predictable, and it is tempting to point a neural network at it and call the result pitch prediction. Most projects that do this blur three different problems: learning a pitcher’s pitch mix, learning count-dependent tendencies, and learning genuine sequencing strategy. A model can score well on the first two while learning little of the third.

I built pitch-sequencing as a lab to take those apart. It generates synthetic pitch data with known structure, puts seven model families behind one interface, and benchmarks them with cross-validation, bootstrap confidence intervals, and paired tests. The goal is decomposition: separate the sources of predictability and measure which models actually use each. The task here is narrow on purpose. It predicts the next pitch type before the pitch is thrown. It does not predict location, velocity, the swing, or the outcome.

What makes this hard

Pitch choice is contextual (count, runners, score), sequential (what was thrown shapes what comes next), adversarial (the batter is adjusting too), and probabilistic. The last word matters most for evaluation: a good model should output a distribution over pitch types, with its uncertainty, instead of a single label. A pitcher who throws 55% fastballs is poorly described by a model that always guesses fastball.

A baseline ladder

Before reaching for a transformer, it helps to climb a ladder of baselines and watch where the gains come from. Each rung adds one source of information: the global pitch rate, then the count, then the pitcher archetype, then count and archetype together, then the previous pitch (a first-order Markov step), then the last two pitches.

The pitcher archetype is the largest single improvement in log loss, while the count alone adds almost nothing over the global rate. The sequence history, the previous one or two pitches, is what lifts top-1 accuracy. And the total gap above the global-rate baseline is small: on this kind of data a simple baseline is hard to beat, and much of a model’s apparent skill is just pitch frequency and count.

The accuracy trap

Two traps make pitch models look stronger than they are.

The first is that accuracy rewards predicting the majority. A model that learns only pitch mix and count can post a respectable number without learning any sequencing at all. The ladder above measures that split.

The second is how the data is split. Random k-fold can reward a pitcher-identity shortcut: if the same pitchers appear in both training and test, a model can memorize each one’s tendencies. The fix depends on the question you are asking. A held-out-pitcher split tests cold-start generalization to pitchers the model has never seen; a time split tests future games for known pitchers using only past information. Both are sturdier than a random row-level split, and they answer different questions. For a known pitcher, prior history is legitimately available, so pitcher identity is not always leakage.

This trap is not hypothetical. While writing this I found and fixed two such issues in the package itself: the benchmark scaled features on the full dataset before splitting, which lets test statistics bleed into training (the fix was to fit the scaler inside each fold), and the previous-pitch feature was built with a global shift that could carry a pitch across game and at-bat boundaries (it now resets at each at-bat). Catching that is exactly what a benchmarking harness is for.

Calibrate, then rank

Because the output should be a distribution, rank models by log loss or Brier score and check calibration, rather than by accuracy alone. A calibrated model’s predicted probabilities match observed frequencies.

The always-fastball guess and the fitted model can sit close on accuracy while the hard, overconfident guesser is heavily penalized by log loss. Accuracy hides that gap; log loss and calibration surface it.

What the package does

Start with the synthetic generator. It encodes four pitcher archetypes (power, finesse, slider specialist, balanced), count-dependent outcomes, eight multi-pitch sequence strategies, fatigue thresholds, and runners and score context. Because the structure is known, you can ask controlled questions: can a Markov model recover the short sequence dependencies the generator put in, and does a transformer add anything once the count and previous pitch are already features, or is it just memorizing the simulator?

On top of that sit seven model families (logistic regression, random forest, HMM, AutoGluon, LSTM, 1D-CNN, transformer) behind one interface, with k-fold cross-validation, bootstrap confidence intervals, paired t-tests and Cohen’s d, log loss and macro-F1 alongside accuracy, ablation studies, a CLI, and MLflow tracking. Together, these keep the comparison reproducible and the assumptions visible.

What synthetic data is good for

Synthetic data is good for controlled experiments, debugging, teaching, and checking that a model can recover structure you put in by hand. It is not evidence of real MLB performance. A model that aces the simulator has shown that it can learn the simulator.

A real-data roadmap

A real-data version would use Statcast, the pitch-level tracking data behind Baseball Savant, pulled with pybaseball. Statcast includes many post-pitch and outcome fields, so the feature set has to be filtered to what is known before the pitch is thrown. Count, baserunners, score, pitcher, batter, inning, and prior pitches are fair game; pitch movement and plate location, batted-ball quality, run-expectancy deltas, and post-pitch score are not available before the pitch. Then group by game and pitcher and plate appearance, and evaluate on future games or held-out pitchers. The field is already well past synthetic data. Recent preprints train transformers on millions of pitch sequences (arXiv:2602.07030) and optimize pitch sequences counterfactually on Statcast (arXiv:2606.17345). A useful caution comes from a 2026 preprint on pitch-pattern motifs (arXiv:2601.11904): pitch sequences carry stable, grammar-like structure, but that structure has limited association with aggregate outcomes like ERA or wins. Sequences are predictable; predicting them is not the same as understanding baseball.

Try it

Clone and install from source:

git clone https://github.com/jman4162/Baseball-Pitch-Sequence-Prediction.git
cd Baseball-Pitch-Sequence-Prediction
pip install -e ".[all]"   # the [all] extra adds AutoGluon and hmmlearn
pitch-generate     # write a synthetic dataset
pitch-benchmark    # k-fold CV across all seven models, with CIs and paired tests

The package exists to pull apart the sources of predictability, pitch mix, count, fatigue, and sequence memory, and to measure which models use each. When a logistic regression on the count and the previous pitch already captures most of the signal, a bigger model has little left to find. The repo has the generator, the model registry, and the benchmark harness.

The figures come from a small synthetic generator in the spirit of the package rather than from real MLB data; the numbers illustrate the method. The package is open-source and the references are public.

This is an independent analysis I did on my own time, using the public sources cited above. The views are my own and do not represent any current or former employer.