In my first writeup, I observed that Exponent’s Logo transients appeared to be bodged too high, even with their unwarranted and adverse use of 67 deg F initialization (Exponent’s “temperature trick”). In today’s post, I’ve taken a closer look at the seemingly questionable calculation of the transients at 67 deg F, showing that the Patriot transients make sense only if initialization for the transients purporting to show Logo Gauge initialization were not actually initialized at 12.5 psi using the Logo Gauge (as stated and as is the purpose of the diagram). My reverse engineering shows that the Patriot dry transient in Figure 27 only makes sense if the Logo Gauge read 12.81 psi at initialization or if the Master Gauge (not the stated Logo Gauge) was erroneously used for initialization. If I’m correct, this is a very significant error – a botch, rather than a bodge – for which one would expect a prompt corrigendum, if not retraction, of the corresponding calculations. In a postscript to today’s post, I’ve attached a note on conversion from Logo and Non-Logo Gauge scale to correctly calibrated Master Gauge scale.
Figure 27 – Simulations with Logo Initialization
The critical simulations pertain to initialization with the Logo gauge. These simulations yield the transients in Figure 26, 27 and 30, that are used for comparison of “models” and observations. As previously noted, Exponent carried out these simulations at 67 deg F – the temperature most disadvantageous to Patriots. Their reasons for this adverse assumption were poorly supported and, in my opinion, are completely invalid. I’ve also observed a puzzling discrepancy between the transients shown in these figures and calculations using the Ideal Gas Law.
First, here is an annotated version of Wells Report Figure 27, showing dry (solid) and wet (dashed) transients for Colt (blue) and Patriot (red) footballs, each supposedly initialized at 13.0 and 12.5 psig at 67 deg F using the Logo gauge. In each case, I’ve shown my reverse-engineered estimates for the actual pressure reading (Logo Gauge) at initialization in order to yield the reported dry transients: 12.81 psig (Patriot) and 13.26 psig (Colt) – 0.31 and 0.26 psi respectively above the Logo readings said to have been used. In each case, I’ve shown the location of transients displaced downwards by these amounts. I’ve also shown (solid dots) the Logo average half-time (converted to Master scale using Exponent’s formulas), each coinciding with the horizontal lines in the diagram. Below the diagram, I’ll explain these calculations and speculate on how the discrepancy may have occurred.
Figure 1. Annotation of Wells Report Figure 27 as explained in text.
The solid dots showing the average Logo half-time measurements converted to Master scale (both shown at plausible times( confirm that the conversion formulas have been accurately implemented.
I digitized the Colt dry transient (solid steelblue) and fit it with a negative exponential to an asymptote. The corresponding y-intercept is 11.90 psig (Master). The pre-game pressure (Master) at 67 deg F initialization required to yield this pressure at 48 deg F ambient is 12.90 psig (Master), which, in turn, is equivalent to 13.26 psig (Logo) using Exponent’s conversion formula. At the stated initialization pressure of 13.0 psig (Logo) at 67 deg F, the corresponding y-intercept is 11.67 psig, about 0.23 psi lower than the y-intercept in Figure 27. A downward translation of the dry transient by this difference of 0.23 psi yields the 13.0 psig transient shown in thicker blue (together with wet transient a further 0.45 psi lower in dashed blue overlapping the Patriot dry transient).
Similarly, the y-intercept of the digitized Patriot dry transient in Figure 27 is ~11.43 psig. The pre-game pressure (Master) at 67 deg F required to yield this pressure at 48 deg F is 12.47 psig (Master), equivalent to 12.81 psig in Logo scale according to Exponent’s conversion formula, ~0.31 psi greater than the 12.5 psig (Non-Logo) said to have been used. A downward translation of the Patriot dry transient by this difference of 0.31 psi yields the 12.5 psig transient shown in thicker red (together with wet transient a further 0.45 psi lower in dashed red.) Note that the Figure 27 Patriot transient is consistent with it being calculated at 12.5 psig initialization using the Master Gauge but not with 12.5 psig initialization using the stated Logo Gauge. Similarly, the Figure 27 Colt transient is consistent with 12.9 psig initialization using the Master Gauge, but not with 13.0 initialization using the Logo Gauge. This seems like a gross error on Exponent’s part. I’ve attached a detailed script documenting my calculation. Using 71 deg F initialization, all transients move lower still.
Figure 25 – Simulations with Non-Logo Initialization
For comparison, the corresponding calculations for the Non-Logo simulations (Figure 25) are analysed identically below. In this case, the transients in Figure 25 are consistent with Colt calculations based on 13.1 psig (Non-Logo; 13.05 Master), as opposed to the stated 13.0 psig, and Patriot calculations based on 12.66 psig (Non-Logo; 12.61 Master), as opposed to the stated 12.5 psig, with re-calculate transients translated downwards by a corresponding amount as shown below.
Figure 2. Annotation of Wells Report Figure 25 (Non-Logo) as explained in text.
The wet differentials for both Colts and Patriots in this figure (~0.2 psi) are considerably less than the wet differential shown in the Logo diagram (~0.45 psi). For reference, wet differentials (dashed lines) using this larger differential are also shown in this figure. As in Figure 1, the average Non-Logo half-time measurements converted to Master scale are shown (solid dots) at plausible times.
Discussion
Exponent clearly stated that its Figure 27 simulations were initialized with the Logo Gauge rather than the Master Gauge:
In recognition of the remaining uncertainty as to which gauge was used to measure the footballs pre-game and in the interest of completeness, similar tests were run using the Logo Gauge. The Logo Gauge was used to set the pressure of two balls to 12.50 psig (representative of the Patriots) and two balls to 13.00 psig (representative of the Colts). From each set (corresponding to each team), one ball remained dry while exposed to the game temperature and the other was wet.
The above Figure 1 shows that this simply isn’t possible. The correctness of this observation can be confirmed by re-calling that initialization at 67 deg F and 12.5 psig yielded half-time pressures at 48 deg F ambient of ~11.5 psig, a value mentioned on several occasions in the Wells Report and observable in Table 10. The Patriot dry transient in Figure 27 is consistent with initialization using the Master Gauge, but not with initialization using the Logo Gauge, as stated and as supposedly the point of the comparison.
This seems like a botch, rather than a bodge, on Exponent’s part and, if so, ought to require a corrigendum, if not retraction.
These differences of 0.3 psi may not seem like much, but the amount in dispute is only ~0.3 psi they are highly material. Also note that any lowering of the transients typically increases the time window in which observations are consistent with transients. Using 71 deg F initialization, all transients move lower still.
Postscript: Conversion to Master Gauge
Before comparing “models” to observations, there are some thorny instrumentation problems that need to be addressed first. Not only was the Logo Gauge biased relative to the Non-Logo Gauge, but it experienced very large drift during Exponent’s studies. Reading between the lines of the report, it looks like they didn’t think about intra-study drift until after it had occurred, very much complicating and somewhat, in my opinion, compromising their conversion of gauge readings to correct (Master Gauge) pressures.
As a preamble, the gap between Logo and Non-Logo gauge measurements at half-time and post-game averaged ~0.38 psi, with no discernible trend between 10.5 and 13.25 psig – see the black +-signs in the figure at right below (overplotted onto an excerpt from Wells Report Figure 12.) This ~0.38 psi Logo bias can be seen in Figure 11 of the Wells Report (see excerpt in left panel below) as the bias at 13 psi in the first (Initial – lightblue) tests; the corresponding test for the Non-Logo Gauge showed no bias. The left panel diagram shows that the Logo Gauge bias had increased to ~0.75 psi by the time of the Final test, with the Non-Logo Gauge now biased low by ~0.15 psi.
Figure 12 of the Wells Report (excerpt in right panel) shows two calibrations (V1, V2) over the Logo and Non-Logo gauges over pressure ranges. Both the V1 and V2 calibrations show considerably larger differences between Logo and Non-Logo readings than actually observed on Game Day, and thus even the V1 tests appear to already experience drift from Game Day readings.
Exponent presented formulas for conversion from Logo and Non-Logo scales to Master scale, with the formulas said to have been calculated from an “early” set of measurements. The differences from Master scale arising from these formulas is shown as the dotted black (Logo) and dotted green (Non-Logo) lines in the right panel. The gap between the lines given by the formulas is noticeably less than the consistent ~0.38 psi observed gap on Game Day, strongly suggesting that some drift had already taken place by the time that Exponent calculated its calibration formulas. The failure of their conversion formulas to preserve this observed difference is a real count against this aspect of their technical work and a frustration to comparative analysis.
Figure 3. Left – excerpt from Wells Report Figure 11, showing Logo and Non-Logo gauge readings in successive tests: note drift upwards of Logo Gauge. Right – excerpt from Wells Report Figure 12. See text for further explanation.
Update: August 5, 2015
he error discussed in this post was also discussed by Back Picks here: http://www.backpicks.com/2015/07/25/deflategate-exponents-bias-and-the-master-error/. They illustrated Exponent’s error: