Every textbook on design of experiments tells you to randomize the run order, and every bioprocess engineer who has tried to randomize bioreactor temperature across sixteen runs has quietly given up somewhere around run five. Resetting a vessel suite to a new temperature and letting it re-equilibrate can cost half a shift. Randomizing it properly can mean doing that a dozen times. The split plot design DOE approach exists for exactly this situation: it lets you change the expensive factor rarely and the cheap factors freely, and it tells you honestly what that trade costs you statistically. This guide covers the whole plot and sub-plot structure, the two error terms that follow from it, a fully worked bioreactor example where the naive analysis inverts the conclusion, and how many whole plots you need before the hard-to-change factor is worth testing at all.
The hard-to-change problem
Complete randomization assumes every run is independently set up from scratch, and for at least one factor in most bioprocess experiments that assumption is false. The design does not become invalid when you break it. It becomes a different design, and it needs a different analysis.
Antony puts the motivation plainly: experimenters sometimes meet situations where randomization of experimental trials is difficult due to cost and time constraints, and temperature in a chemical process is his example of a hard-to-change factor that makes complete randomization almost impossible. His recommendation is to change the levels of that factor less frequently than the others, which is restricted randomization. A split-plot design is restricted randomization given a formal structure and, crucially, a matching analysis.
The cost of ignoring the problem is concrete. Consider a 16-run full factorial design in three factors, replicated twice, with eight runs at each of two temperature setpoints. If you randomize the run order completely, the expected number of times temperature changes between consecutive runs is exactly 8.0. A split-plot version with four whole plots requires four temperature setups. That is not a marginal saving. It is often the difference between a study that gets executed and one that gets abandoned after week two.
Two responses to this are both wrong. The first is to run the fully randomized design on paper and then quietly group the temperature runs during execution, without telling the analysis. The second is to run temperature in blocks and analyse the result as though it had been fully randomized. Both produce the same statistical failure, and it is the one described in detail below.
Whole plots and sub-plots
A whole plot is the group of runs that share one setting of the hard-to-change factor. A sub-plot is a single run inside that group. Randomization happens twice: once to decide the order of the whole plots, and once, independently, inside each whole plot.
The whole plot subplot vocabulary comes from agriculture, where a tractor ploughed a whole field one way and smaller strips within it received different seed varieties. The bioprocess translation is direct. A campaign day on a four-vessel bench bioreactor suite sharing one temperature-controlled circulator is a whole plot. The four vessels running that day, each with its own feed rate and pH setpoint, are the sub-plots.
Figure 1 shows four whole-plot boxes representing four campaign days at temperatures 33, 36.5, 33 and 36.5 degrees Celsius. Each box contains four sub-plot cells covering the four combinations of high and low feed rate with high and low pH, arranged in a different random order in each box. Annotations identify whole-plot error with two degrees of freedom as variation between days at the same temperature, and sub-plot error with six degrees of freedom as variation between vessels within a single day.
Notice what the whole plot absorbs. Everything that is shared by the four vessels on a given day rides along with it: the inoculum from one seed train, the medium lot opened that morning, the operator, the ambient conditions, the calibration state of the analyzer. That is why whole-plot variation is nearly always larger than sub-plot variation, and it is the physical reason the two error terms differ. Careful blocking and a well-controlled seed train shrink whole-plot error but never eliminate it.
Lay out the design before you book the reactors
Build the factorial, randomize the run order, and see the run count before you commit a suite to a four-day campaign.
Hard to change factors in DOE across bioprocess
A factor is hard to change when resetting it between consecutive runs costs more than grouping the runs costs you in statistical power. That is a judgement about your facility, not a property of the variable, and it is worth making explicitly before the design is fixed.
Antony's checklist for deciding is short and practical: what is the cost associated with changing factor levels, what is the setup time between trials, how many factors are expensive or difficult to control, and where do you assign factors whose levels are difficult to change. Run that checklist over your factor list and the split into whole-plot and sub-plot candidates usually falls out immediately.
| Factor | Reset cost between runs | Why | Assign to |
|---|---|---|---|
| Bioreactor temperature | Hours | Vessel and jacket must re-equilibrate; often one circulator serves several vessels | Whole plot |
| Medium or feed lot | Days to weeks | Minimum order quantity; lots arrive in batches and cannot be alternated per run | Whole plot |
| Inoculum train / seed age | Days | One seed train feeds a whole vessel group in the same expansion | Whole plot |
| Sparger or impeller configuration | Hours, plus a break in containment | Requires vessel teardown and re-sterilisation | Whole plot |
| Feed rate / feed profile | Minutes | Pump setpoint, independent per vessel | Sub-plot |
| pH setpoint | Minutes | Controller setpoint, independent per vessel | Sub-plot |
| Dissolved oxygen setpoint | Minutes | Controller setpoint, though shared gas supply can couple vessels | Usually sub-plot |
| Supplement concentration | Minutes | Bolus addition per vessel | Sub-plot |
One row deserves a caveat. Dissolved oxygen looks like a sub-plot factor because it is a controller setpoint, but if the vessels share a gas supply manifold or a single mass flow controller, changing one vessel's DO strategy can perturb its neighbours. In that case DO is coupled across the group and belongs in the whole plot. The test is not "can I type a different number", it is "can I set this vessel's level without touching the others".
Two error terms, and why people get this wrong
Because randomization happened twice, there are two sizes of experimental unit and therefore two experimental errors. Each factor must be tested against the error of the unit its level was randomized to. There is no single error term that serves the whole design.
The hard-to-change factor was randomized to whole plots, so it is tested against whole-plot error. The easy-to-change factors and every interaction that involves one of them were randomized to sub-plots, so they are tested against sub-plot error. That includes the interaction between the hard-to-change factor and an easy one, which is a pleasant surprise the first time you meet it: the temperature by feed interaction is a sub-plot effect and is estimated precisely, even though the temperature main effect is not.
The degrees of freedom split accordingly. For a design with a whole plots and n sub-plots each:
of which: hard-to-change factor df, remainder = whole-plot error
sub-plot stratum df = a(n − 1)
of which: sub-plot terms, remainder = sub-plot error
With four whole plots and four sub-plots each, the whole-plot stratum has 3 df, of which temperature takes 1 and whole-plot error keeps 2. The sub-plot stratum has 12 df, of which the six sub-plot terms take 6 and sub-plot error keeps 6. That is a total of 15, as it must be for 16 runs.
Two degrees of freedom is not much, and that is the real price of the split-plot. The critical F value at α = 0.05 with 1 and 2 degrees of freedom is 18.51, against 5.99 with 1 and 6. You are testing the factor you probably care most about with the weakest test in the whole analysis.
Figure 2 contrasts two analyses of the same data. The correct split-plot analysis tests temperature against whole-plot error with mean square 0.390 on 2 degrees of freedom, giving F equal to 2.77 against a critical value of 18.51, which is not significant; and tests feed, pH and all interactions against sub-plot error with mean square 0.0185 on 6 degrees of freedom against a critical value of 5.99. The incorrect analysis pools everything into one residual with mean square 0.1115 on 8 degrees of freedom. Because 0.0185 is less than 0.1115 which is less than 0.390, the pooled error is too small for temperature and produces a false positive at p equal to 0.014, and too large for pH and the feed by pH interaction, which are missed.
This is the single most important fact about split-plot analysis, and it is why pooling is not a safe conservative shortcut. Pooling is conservative for one stratum and anti-conservative for the other, simultaneously. In the example below the pooled mean square is exactly the weighted average of the two: (2 × 0.3904 + 6 × 0.01853) / 8 = 0.1115. It is a real number, but it is not the error term for anything.
Worked example: one dataset, two answers
The same 16 titer measurements support opposite conclusions depending on whether the split-plot structure is declared. Here is the whole thing, from design to ANOVA.
The study
A CHO fed-batch development campaign on a four-vessel bench suite. One temperature-controlled circulator serves all four vessels, so temperature is fixed for a whole campaign day. Feed rate and pH are per-vessel controller settings.
- Whole-plot factor. Temperature after day 3: 33 °C (a production-phase shift) coded −1, or held at 36.5 °C coded +1.
- Sub-plot factor F. Feed rate: 3% of working volume per day coded −1, 5% coded +1.
- Sub-plot factor pH. Setpoint 6.9 coded −1, 7.1 coded +1.
- Response. Harvest titer, g/L.
- Structure. Four whole plots (two campaign days at each temperature), four sub-plots each, N = 16.
| Whole plot | Temp (T) | Feed (F) | pH (P) | Titer (g/L) | Whole-plot mean |
|---|---|---|---|---|---|
| 1 | −1 | −1 | −1 | 3.55 | 3.46 |
| 1 | −1 | −1 | +1 | 2.83 | |
| 1 | −1 | +1 | −1 | 3.74 | |
| 1 | −1 | +1 | +1 | 3.72 | |
| 2 | +1 | −1 | −1 | 2.28 | 2.50 |
| 2 | +1 | −1 | +1 | 2.07 | |
| 2 | +1 | +1 | −1 | 2.76 | |
| 2 | +1 | +1 | +1 | 2.89 | |
| 3 | −1 | −1 | −1 | 2.90 | 3.06 |
| 3 | −1 | −1 | +1 | 2.66 | |
| 3 | −1 | +1 | −1 | 3.45 | |
| 3 | −1 | +1 | +1 | 3.23 | |
| 4 | +1 | −1 | −1 | 2.89 | 2.98 |
| 4 | +1 | −1 | +1 | 2.30 | |
| 4 | +1 | +1 | −1 | 3.29 | |
| 4 | +1 | +1 | +1 | 3.44 |
Look at the whole-plot means before any arithmetic. The two 33 °C days gave 3.46 and 3.06 g/L, and the two 36.5 °C days gave 2.50 and 2.98. The gap between the two temperature groups is real but so is the gap within each group, and it is nearly as big. That is whole-plot error, visible by eye, and it is what any honest test of temperature has to beat.
Effect estimates (coded units)
With an orthogonal two-level design, each effect is the difference between the mean at the high level and the mean at the low level, and each sum of squares is the squared contrast divided by N:
- Temperature: −0.520 g/L (SS = 1.0816)
- Feed rate: +0.630 g/L (SS = 1.5876)
- pH: −0.215 g/L (SS = 0.1849)
- Feed × pH: +0.225 g/L (SS = 0.2025)
- Temp × Feed: +0.080 (SS = 0.0256) · Temp × pH: +0.085 (SS = 0.0289) · three-way: +0.045 (SS = 0.0081)
Total sum of squares = 4.0112 on 15 df. The effect estimates are identical under either analysis. Only the error terms and therefore the p-values differ.
Now partition the sums of squares by stratum. The whole-plot stratum is built from the four whole-plot means: 4 × Σ(mean − 3.00)² = 1.8624 on 3 df. Temperature takes 1.0816 of that on 1 df, leaving whole-plot error = 0.7808 on 2 df, MS = 0.3904. Everything else falls in the sub-plot stratum, where the six sub-plot terms take 2.0376 and sub-plot error = 0.1112 on 6 df, MS = 0.01853.
| Source | SS | df | MS | F | p | Verdict |
|---|---|---|---|---|---|---|
| Whole-plot stratum (tested against MS = 0.3904, 2 df, Fcrit = 18.51) | ||||||
| Temperature | 1.0816 | 1 | 1.0816 | 2.77 | 0.238 | not significant |
| Whole-plot error | 0.7808 | 2 | 0.3904 | — | — | s = 0.625 g/L |
| Sub-plot stratum (tested against MS = 0.01853, 6 df, Fcrit = 5.99) | ||||||
| Feed rate | 1.5876 | 1 | 1.5876 | 85.66 | 0.0001 | significant |
| pH | 0.1849 | 1 | 0.1849 | 9.98 | 0.020 | significant |
| Feed × pH | 0.2025 | 1 | 0.2025 | 10.93 | 0.016 | significant |
| Temp × Feed | 0.0256 | 1 | 0.0256 | 1.38 | 0.284 | not significant |
| Temp × pH | 0.0289 | 1 | 0.0289 | 1.56 | 0.258 | not significant |
| Temp × Feed × pH | 0.0081 | 1 | 0.0081 | 0.44 | 0.533 | not significant |
| Sub-plot error | 0.1112 | 6 | 0.01853 | — | — | s = 0.136 g/L |
| Total | 4.0112 | 15 | — | — | — | — |
The naive analysis is what any software gives you if you feed it the eight columns and never mention the whole plots. It fits all seven terms and keeps a single residual of 0.7808 + 0.1112 = 0.8920 on 8 df, mean square 0.1115. Test everything against that and temperature comes out at F = 9.70, p = 0.014, comfortably significant, while pH sits at p = 0.234 and the feed by pH interaction at p = 0.215, both discarded.
Read that chart carefully, because the practical damage is easy to understate. A team acting on the naive analysis would rewrite the temperature setpoint in the process description on the strength of an effect the data cannot actually support, and would leave pH and the feed by pH interaction out of the model entirely. The next round of response surface work would then optimize the wrong two factors.
It is worth being precise about what the split-plot analysis is and is not saying about temperature. It is not saying the effect is zero. The observed −0.52 g/L is a perfectly reasonable point estimate, and it is consistent with what temperature-shift strategies do to CHO titer. It is saying that with only two whole plots per temperature level, this experiment cannot distinguish that effect from ordinary day-to-day variation. That is a statement about the design, not about the biology, and the fix is more whole plots rather than a different test.
Check the run count before you lock the design
Generate the factorial, add replicates, and get a randomized run order you can hand to the suite.
How many whole plots do you actually need?
Whole-plot replication, not total run count, is what buys power on the hard-to-change factor. Adding vessels inside each whole plot does almost nothing for it.
This is the most counter-intuitive property of the design and the one that catches people out. Doubling the number of sub-plots per whole plot doubles your run count, doubles your analytics burden, and sharpens every sub-plot effect. It barely moves the test on temperature, because temperature's degrees of freedom depend only on the number of whole plots. With a whole plots and a two-level hard-to-change factor, whole-plot error has a − 2 degrees of freedom. Four whole plots gives 2.
Using the variance components estimated above (sub-plot σ = 0.136 g/L, whole-plot σ = 0.305 g/L), the smallest effect detectable at α = 0.05 works out as follows.
| Whole plots | Total runs | Whole-plot error df | Detectable temperature effect | Detectable sub-plot effect |
|---|---|---|---|---|
| 4 | 16 | 2 | 1.34 | 0.17 |
| 6 | 24 | 4 | 0.71 | 0.12 |
| 8 | 32 | 6 | 0.54 | 0.10 |
| 10 | 40 | 8 | 0.46 | 0.09 |
| 12 | 48 | 10 | 0.40 | 0.08 |
| 16 | 64 | 14 | 0.34 | 0.07 |
| 20 | 80 | 18 | 0.29 | 0.06 |
The first row explains the worked example completely. At four whole plots the smallest detectable temperature effect is 1.34 g/L on a process running at 3.00 g/L. The effect present was 0.52 g/L. The experiment was never capable of proving it, and the split-plot analysis is simply reporting that fact. The naive analysis appeared to prove it only because it was using a yardstick 3.5 times too short.
The practical guidance follows directly. If the hard-to-change factor is genuinely the question, budget six to eight whole plots and accept the campaign length. If the hard-to-change factor is a nuisance you must span rather than a factor you must resolve, four whole plots is fine, since the sub-plot effects are already resolved to 0.17 g/L there. What you must not do is run four whole plots, get a small p-value from the wrong error term, and believe it. Sizing this properly is the same exercise as any other DOE run-count decision, just applied separately to each stratum.
Split-plot vs completely randomized
A split-plot buys execution feasibility and sub-plot precision, and pays for it with whole-plot precision. It is a genuine trade, not a compromise you should feel bad about.
| Property | Completely randomized | Split-plot |
|---|---|---|
| Temperature setups required | 8 on average | 4 |
| Error terms | One | Two (whole-plot and sub-plot) |
| Precision, hard-to-change factor | Higher, if you can execute it | Lower, limited by whole-plot count |
| Precision, easy-to-change factors | Baseline | Higher, whole-plot noise removed |
| Hard × easy interactions | Baseline | Estimated in the sub-plot stratum, so precise |
| Analysis | Standard ANOVA | Mixed model or two-stratum ANOVA |
| Main risk | Never actually gets executed as designed | Gets executed, then analysed as if it were CRD |
The row worth dwelling on is the interaction row. Because interactions between the hard-to-change factor and the easy ones live in the sub-plot stratum, a split-plot estimates them better than a completely randomized design of the same size would. If your real question is "does the optimal feed rate depend on temperature", a split-plot is not a concession at all. It is the more efficient design.
There is also a scale-up angle. Whole plots often correspond to something physically meaningful, such as one vessel or one campaign, so the whole-plot error is a direct estimate of the run-to-run variability you will face in manufacturing. That number is worth having on its own, and it feeds straight into scale-up calculations and into any argument about how many engineering runs a tech transfer needs.
Restricted randomization in practice
Restricted randomization DOE means deliberately limiting the run order so an expensive factor changes rarely, and then declaring that restriction in the analysis. The declaring is the part people skip.
Antony frames the choice in terms of process stability. If you believe your process is stable, you do not need to randomize. If you believe it is unstable and results without randomization would be meaningless, you must randomize. And if the process is so unstable that randomization makes the experiment impossible, do not run the experiment at all, and go and bring the process into statistical control first. A split-plot sits in the middle of that spectrum: it randomizes as much as the facility permits and accounts honestly for what it could not randomize.
Executing one comes down to a short checklist.
- Declare the whole-plot factor before the design is built. Retro-fitting the structure after the runs are done is guesswork about which runs shared what.
- Randomize the order of the whole plots. Do not run both low-temperature days first. That confounds temperature with campaign drift exactly as described in randomization and replication in DOE.
- Randomize inside each whole plot independently. A fresh random order per whole plot, not the same order repeated.
- Record the whole-plot identifier in the data table. This is the one column that makes the correct analysis possible later, and the one most often missing.
- Replicate whole plots, not just sub-plots. Two whole plots per level is the minimum that yields any whole-plot error at all, and it yields only 2 df.
- Analyse with a mixed model, or by hand with two strata. Whole plot must enter as a random effect. If your software only offers standard ANOVA, the two-stratum arithmetic in Table 3 is a few lines of work.
Point four deserves emphasis because it is cheap insurance. Even if you intend a fully randomized design, recording which runs shared a setup, a medium lot, or a day costs nothing at execution time and lets you check afterwards whether the structure crept in anyway. It very often did.
The last thing to say is that a split-plot is not a lesser experiment. It is what an honest DOE for cell culture and fermentation usually has to look like, because bioreactors do not reset instantly and seed trains do not branch on demand. Build the factorial in a free DOE software tool, decide which factors go in the whole plot, replicate the whole plots properly, and record the structure. The design will then tell you the truth about what it can and cannot resolve, which is the only thing worth asking of it.
Frequently Asked Questions
What is a split-plot design in DOE?
A split-plot design is a designed experiment containing at least one factor that is too expensive or slow to reset for every run. That hard-to-change factor is held constant across a group of runs called a whole plot, and the easy-to-change factors are randomized independently within each whole plot in units called sub-plots. Because randomization happens at two levels, the design has two error terms rather than one, and each factor must be tested against the error term belonging to its own stratum.
What is a hard to change factor in DOE?
A hard-to-change factor is one you cannot independently reset between consecutive runs without unacceptable cost or delay. In bioprocessing the usual examples are bioreactor temperature, a shared gassing or agitation configuration, medium lot, inoculum train, and any setting that requires re-equilibrating a vessel for hours. Antony gives temperature in a chemical process as the classic case and notes that complete randomization of such a factor is almost impossible, so the level should be changed less frequently than the others.
Why does a split-plot design have two error terms?
Because there are two separate acts of randomization and therefore two sizes of experimental unit. The whole-plot error measures how much whole plots at the same hard-to-change setting differ from one another, and it is the correct yardstick for the hard-to-change factor. The sub-plot error measures how much runs inside the same whole plot differ, and it is the correct yardstick for the easy-to-change factors and every interaction involving them. Whole-plot error is almost always the larger of the two because a whole plot absorbs day-to-day, lot-to-lot, and setup variation that sub-plots share.
What happens if you analyze a split-plot as a completely randomized design?
You make two errors at once and they point in opposite directions. The pooled residual sits between the two true error terms, so it is too small for the hard-to-change factor, which becomes over-significant, and too large for the easy-to-change factors, which get missed. In the worked 16-run example on this page the naive analysis reported temperature as significant at p = 0.014 when the correct split-plot test gives p = 0.238, and it missed both the pH effect and the feed by pH interaction that the correct analysis finds at p = 0.020 and p = 0.016.
How many whole plots does a split-plot design need?
More than most people run. With a two-level hard-to-change factor, the whole-plot error has only (number of whole plots minus 2) degrees of freedom, so four whole plots leave just 2 df and a critical F of 18.51. In the worked example that puts the smallest detectable temperature effect at about 1.34 g/L, far larger than the 0.52 g/L actually present. Eight whole plots bring that down to about 0.54 g/L. Plan six to eight whole plots if the hard-to-change factor genuinely matters, and accept that the sub-plot factors are well resolved either way.
What is restricted randomization in DOE?
Restricted randomization is any deliberate limit on the run order that stops a factor level from changing on every run. Antony recommends it precisely where cost and time make complete randomization impractical, and a split-plot design is its most structured form: randomize the order of the whole plots, then randomize the runs inside each whole plot independently. The important point is that the restriction must be declared in the analysis. A restriction you impose during execution but ignore in the ANOVA is what produces a false positive on the hard-to-change factor.
Related Tools
- DOE Experiment Generator — Build the factorial, set replicates, and get the randomized run order to split into whole plots.
- Scale-Up Calculator — Turn the whole-plot error estimate into a realistic view of run-to-run variability at the next scale.
- Fed-Batch Calculator — Work out the feed profiles that the sub-plot levels actually correspond to at the bench.
References
- Jones, B. & Nachtsheim, C.J. (2009). Split-Plot Designs: What, Why, and How. Journal of Quality Technology, 41(4), 340–361. doi:10.1080/00224065.2009.11917790
- Wooding, W.M. (1973). The Split-Plot Design. Journal of Quality Technology, 5(1), 16–33. doi:10.1080/00224065.1973.11980566
- Antony, J. (2003). Design of Experiments for Engineers and Scientists. Butterworth-Heinemann/Elsevier. ISBN 0-7506-4709-4. (§2.2.1 randomization and restricted randomization for hard-to-change factors; §8.1.8 randomize the experimental trial order.)
- Montgomery, D.C. (2017). Design and Analysis of Experiments, 9th ed. Wiley. ISBN 978-1119113478. (Split-plot and nested designs, and the two-error-term analysis.)
- Mandenius, C.-F. & Brundin, A. (2008). Bioprocess optimization using design-of-experiments methodology. Biotechnology Progress, 24(6), 1191–1203. doi:10.1002/btpr.67