Keane Hauck

Why the F-statistic is equivalent to the squared t-statistic for a two-group comparison

Mon, 09 Mar 2026 00:00:00 +0000

Possibly the most horrifying seven words you can encounter when reading a textbook are the following: “The proof is left to the reader.” When learning ANOVA (Analysis of Variance), it is common to encounter the fact that the F-statistic is equal to the squared t-statistic when you are only comparing two groups. Unfortunately, this sentence is often followed by the previously-mentioned proof hand-waving, which can leave readers confused, myself included. Thus, I am going to clearly and concisely demonstrate how the F-test reduces to a t-test under certain conditions.

Interestingly, this proof is quite elusive. In fact, while researching how to write this up, the only published proof I stumbled across was another blog post from a data scientist named Juan Manuel. Big shoutout to him for his post, which helped me double-check my results.

This proof assumes familiarity with the ANOVA and t-test framework.

Setup

In the notation we’ll be using,

$SS_B$: Sum of Squares between groups due to treatment
$SS_W$: Sum of Squares within groups due to error
$MS_B$: Mean Sum of Squares between groups $=SS_B / k-1$
$MS_W$: Mean Sum of Squares within groups $=SS_W / N-k$
$N$: Total number of observations
$n_i$: Number of observations in group $i$
$k$: Number of groups
$j$: Index of a single observation
$\overline{y}$: The grand mean
$\overline{y}_i$: Mean of group $i$

A few formulae are important for the following results:

$t=\frac{\overline{y_1}-\overline{y_2}}{SE_{\overline{y_1}-\overline{y_2}}}$

where $SE_{\overline{y_1}-\overline{y_2}}=s_p \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}$.

Sample variance: $s_i^2=\frac{\sum_{j=1}^{n_i}(y_{ij}-\overline{y}_i)^2}{n_i-1}$ for group $i$.

Pooled sample variance for two groups: $s^2_p = \frac{s^2_1(n_1-1) \ + \ s^2_2(n_2-1)}{N-2}$.

$F = \frac{MS_{\text{B}}}{MS_{\text{W}}}$

with $MS_B= \frac{\sum_{i=1}^{k} n_i (\overline{y}_i - \overline{y})^2}{k-1}$

and $MS_{\text{W}} = \frac{ \sum_{i=1}^{k} \sum_{j=1}^{n_i} \left(y_{ij} - \overline{y}_i \right)^2 }{N-k}$.

This final $F$ equation is the one we need to reduce to obtain $t^2$.

Simplifying Denominator

The first step is to figure out what the denominator of the expression simplifies to. With two groups, $MS_{W}=\frac{SS_{W}}{N-2}$.

Again, with two groups,

$SS_W= \sum_{i=1}^{k} \sum_{j=1}^{n_i}(y_{ij} - \overline{y}_i)^2=\sum_{j=1}^{n_1}(y_{1j}-\overline{y}_1)^2 + \sum_{j=1}^{n_2}(y_{2j}-\overline{y}_2)^2$.

Thus, $MS_{W}= \frac{\sum_{i=1}^{k} \sum_{j=1}^{n_i}(y_{ij} - \overline{y}_i)^2}{N-k}=\frac{\sum_{j=1}^{n_1}(y_{1j}-\overline{y}_1)^2\ +\ \sum_{j=1}^{n_2}(y_{2j}-\overline{y}_2)^2}{N-2}$ for two groups.

The goal is to represent this equation in terms of something we recognize. We know that sample variance, $s_i^2$, is equal to $\frac{\sum_{j=1}^{n_i}(y_{ij}-\overline{y}_i)^2}{n_i-1}$ for group $i$. We might notice that the numerator of this equation shows up twice in our current $MS_W$ formula. We can make sample variance itself show up by multiplying both the numerator and denominator of $MS_W$ by $\frac{1}{(n_1-1)(n_2-1)}$.

$MS_W \cdot \frac{\frac{1}{(n_1-1)(n_2-1)}}{\frac{1}{(n_1-1)(n_2-1)}}=\frac{\sum_{j=1}^{n_1}(y_{1j}-\overline{y}_1)^2\ +\ \sum_{j=1}^{n_2}(y_{2j}-\overline{y}_2)^2}{N-2} \cdot \frac{\frac{1}{(n_1-1)(n_2-1)}}{\frac{1}{(n_1-1)(n_2-1)}}$

$= \frac{\sum_{j=1}^{n_1}(y_{1j}-\overline{y}_1)^2/(n_1-1)(n_2-1)\ +\ \sum_{j=1}^{n_2}(y_{2j}-\overline{y}_2)^2/(n_1-1)(n_2-1)}{(N-2)/(n_1-1)(n_2-1)}=\frac{s^2_1 / (n_2-1) \ + \ s^2_2 / (n_1-1)}{(N-2)/(n_1-1)(n_2-1)}$.

This is starting to look a lot like the pooled variance estimator for two groups, $s^2_p = \frac{s^2_1(n_1-1) \ + \ s^2_2(n_2-1)}{N-2}$. If we multiply our current equation by $\frac{(n_1-1)(n_2-1)}{(n_1-1)(n_2-1)}$ we obtain exactly this formula:

$\frac{s^2_1 / (n_2-1) \ + \ s^2_2 / (n_1-1)}{(N-2)/(n_1-1)(n_2-1)} \cdot \frac{(n_1-1)(n_2-1)}{(n_1-1)(n_2-1)} = \frac{s^2_1(n_1-1) \ + \ s^2_2(n_2-1)}{N-2} = s^2_p$.

Thus, we have proven the denominator of the two-group $F$ formula is equal to the pooled variance estimator.

Simplifying Numerator

The numerator, $MS_B$, is a little trickier. We know that $t=\frac{\overline{y_1}-\overline{y_2}}{s_p \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$ and $t^2= \frac{(\overline{y_1}-\overline{y_2})^2}{s^2_p (\frac{1}{n_1}+\frac{1}{n_2})}$. If our denominator is equivalent to $s^2_p$, the numerator must equal $\frac{(\overline{y_1}-\overline{y_2})^2}{ (\frac{1}{n_1}+\frac{1}{n_2})}$ for our overall formula to equal $t^2$. Let’s prove it.

$MS_B= \frac{\sum_{i=1}^{k} n_i (\overline{y}_i - \overline{y})^2}{k-1}$. For two groups, $k-1=1$ and this equation simplifies to $MS_B= n_1(\overline{y}_1 - \overline{y})^2\ + \ n_2(\overline{y}_2 - \overline{y})^2$ by expanding the sum for $k=2$.

We can represent the grand mean $\overline{y}$ as a weighted average $\overline{y}= \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N}$ with $N=n_1 + n_2$.

If we substitute it into our equation for two groups, we get $MS_B= n_1(\overline{y}_1 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 + n_2(\overline{y}_2 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2$.

This equation consists of two similar effect terms, one for each group. We will now focus on one of these terms to simplify it, with the logic that the other term will simplify in the same manner. Let’s examine $n_1(\overline{y}_1 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2$.

We will now make a few algebraic steps. Each should be clear and followable.

\[\begin{align*} n_1(\overline{y}_1 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 &= n_1(\frac{\overline{y}_1N}{N} - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 \\ &= n_1(\frac{n_1 \overline{y}_1+n_2 \overline{y}_1}{N} - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 \ \ \ (\text{since} \ N=n_1+n_2) \\ &= n_1(\frac{n_2 \overline{y}_1 - n_2 \overline{y}_2}{N})^2 \\ &= n_1(\frac{n_2 (\overline{y}_1 - \overline{y}_2)}{N})^2 \\ &= n_1(\frac{n_2^2 (\overline{y}_1 - \overline{y}_2)^2}{N^2}) \\ &= \frac{n_1n_2^2}{N^2}(\overline{y}_1 - \overline{y}_2)^2. \end{align*}\]

Thus, our first term $n_1(\overline{y}_1 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2$ is equivalent to $\frac{n_1n_2^2}{N^2}(\overline{y}_1 - \overline{y}_2)^2$. The same process can be repeated for the second term (switching the $n$’s and $\overline{y}’s$ as appropriate), which obtains $n_2(\overline{y}_2 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 = \frac{n_2n_1^2}{N^2}(\overline{y}_2 - \overline{y}_1)^2$.

So, our whole equation for $MS_B$ can be rewritten as $MS_B = n_1(\overline{y}_1 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 + n_2(\overline{y}_2 - \frac{n_1 \overline{y}_1 + n_2 \overline{y}_2}{N})^2 = \frac{n_1n_2^2}{N^2}(\overline{y}_1 - \overline{y}_2)^2 + \frac{n_2n_1^2}{N^2}(\overline{y}_2 - \overline{y}_1)^2$.

By definition, $(\overline{y}_1 - \overline{y}_2)^2 = (\overline{y}_2 - \overline{y}_1)^2$, so the entire equation simplifies to $[\frac{n_1n_2^2}{N^2} + \frac{n_2n_1^2}{N^2}](\overline{y}_1 - \overline{y}_2)^2$.

We will conduct one final sequence of algebraic steps:

\[\begin{align*} [\frac{n_1n_2^2}{N^2} + \frac{n_2n_1^2}{N^2}] (\overline{y}_1 - \overline{y}_2)^2 &= [\frac{n_1n_2^2+n_2n_1^2}{N^2}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{n_1n_2n_2+n_2n_1n_1}{N^2}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{n_1n_2(n_1+n_2)}{N^2}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{n_1n_2(n_1+n_2)}{(n_1+n_2)^2}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{n_1n_2}{n_1+n_2}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{1}{\frac{n_1+n_2}{n_1n_2}}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{1}{\frac{n_1}{n_1n_2}+\frac{n_2}{n_1n_2}}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= [\frac{1}{\frac{1}{n_2}+\frac{1}{n_1}}] (\overline{y}_1 - \overline{y}_2)^2 \\ &= \frac{(\overline{y}_1 - \overline{y}_2)^2} {\frac{1}{n_2}+\frac{1}{n_1}}. \end{align*}\]

Thus, for two groups our numerator $MS_B$ is equivalent to $\frac{(\overline{y}_1 - \overline{y}_2)^2}{\frac{1}{n_2}+\frac{1}{n_1}}$.

Conclusion

The F-statistic is the ratio of $\frac{MS_B}{MS_W}$. We have shown that for two groups ($k=2$) this is equivalent to the ratio $\frac{(\overline{y}_1 - \overline{y}_2)^2}{s^2_p(\frac{1}{n_2}+\frac{1}{n_1})}$, which is exactly equal to $t^2$.

And there you have it.

P.S.

There are obviously multiple ways of proving this result. A much simpler version is in the general linear model framework where you can show that the test of the coefficient for a single-predictor (like group membership) model follows a $t$ distribution with $t_{obs}=\frac{b_1}{s_{b_1}}$. It can then be shown that squaring this is equivalent to the observed $F$ value. However, I wanted to highlight a method of proof that only requires familiarity with the ANOVA framework and some solid algebra skills. I hope this is helpful for anyone like me who was curious about this often-stated fact about hypothesis testing.

24 Hour Planet Survival Challenge

Sat, 31 Jan 2026 00:00:00 +0000

So I was thinking about something as I fell asleep last night. I was reading about the environmental conditions of Venus—surprise: it’s incredibly hazardous and unsuitable for life. At sea level, the atmospheric pressure is about 93x that of Earth, and the temperature averages around 850 °F. Obviously any human that found themselves transported to the surface of Venus would be immediately obliterated. But this begs the question—what planets in the solar system are the most hospitable to human life? The least?

Thus I came up with an idea for a challenge. If I hypothetically had access to unlimited materials and a year’s worth of time, how difficult would it be for me to build a contraption that would allow me to survive on each planet in the solar system for 24 hours? I’m going to step through my process of hypothetical survival and rank each planet in terms of its survivability.

Ground rules

A few basic rules are in order.

It’s assumed that I can eventually build each contraption.

Obviously I know nothing about building random machines. However, with unlimited access to materials (including money) and a full calendar year, I would be able to contract with different manufacturing companies to theoretically construct whatever I wanted. We could also just assume that I had a giant 3D printer that could spit out whatever machine I wanted. Either way, this is the least interesting part of the challenge so I’m just going to make it a given.

The contraptions need to be as simple as possible.

It’s totally uninteresting to just surround myself with a 1000-foot thick ball of steel and plop it on the surface of some random planet (According to the heat equation a 1000-foot thick ball of steel will definitely maintain a stable interior temperature for 24 hours. However, we can do better. The thermal diffusivity chart on wikipedia suggests that steel is a rather poor medium for our giant ball. Rubber actually fares really well, followed by glass). Therefore, the challenge is as interesting as possible if our goal is to make a simple, exposed contraption.

My contraption and I will be teleported to surface level of the planet (if it has a surface)

Similarly to the last point, it’s kind of cheating to say I can just float 5 miles above the surface, where no storms/gravity/temperatures can reach me. At the logical extreme, what’s to stop me from just orbiting a planet for 24h? This rule ensures I have to at least survive at a reasonable altitude. If the planet is a gas giant like Jupiter, we’ll discuss what seems fair as we get to it.

Ok, those are the rules! I’ll go through the ranking from most survivable to least survivable.

1. Mars

Perhaps the biggest shock of this entire ranking is that Earth isn’t the most survivable planet. The crux of the matter boils down to this: Earth has surface water, while Mars does not. I have to account for the fact that about 71% of Earth’s surface is covered with water, so if I’m teleported to a random location at surface level there’s a good chance I end up in one of the oceans. Mars, on the other hand, is predictable. Most of its surface is dry, cold, and rocky, which makes it much simpler to figure out how to survive. NASA has developed theoretical suits for walking on Mars and they have oxygen tanks rated for longer than 24 hours, so all I would need to do is contact NASA to build one for me and train me how to use it. Not only is it totally survivable (barring suit malfunctions), but this scenario might end up being the coolest trip of my entire life: walking around Mars and exploring the red landscape for 24 hours. Unbelievably, the biggest issue might be how I would use the bathroom.

2. Earth

Mother Earth really let us down here. If I could pick and choose the location I’m being teleported to, there would be no problem at all. Factoring for the oceans creates some problems. My contraption could be a cruise ship, but is this necessary? I think we can do this without even needing a boat of any sort. The SOLAS standards for life jacket buoyancy require them to lose less than 5% buoyancy over the span of 24 hours, so we’re all good there. The bigger threat would be the temperature. A large percentage of the ocean has temperatures below freezing, so I would need to bundle up. We could get technical with the temperature ratings of different wetsuits, but I think there’s an easier solution.

An atmospheric diving suit has just about everything I’d need to survive 24h in the ocean: thruster maneuverability, a maintained internal atmospheric pressure/temperature, and a 24-hour emergency backup oxygen tank. So, I’d just secure a bunch of life jackets to one of those suits. If the lifejackets fail, I should be able to jet around with the thrusters to avoid getting too deep. I can also use my year of time to learn how to maneuver my suit. And in the off-chance I get teleported to land, I can just walk around in my modern suit of armor to avoid any natural predators. Easy survival.

3. Mercury

The World Sauna Championships was an annual “heat-endurance” event held in Finland from 1999 to 2010. Each year, contestants would sit in an extremely hot sauna until they gave up, with the winner being the last person to be able to walk out of the sauna without outside assistance. Temperatures typically started at 110°C. In 2010, the event was permanently cancelled after a competitor unfortunately passed away due to complications from extreme heat. He had sat in the sauna for only 6 minutes. The moral? Surviving extreme heat is really difficult.

This is the problem we run into with Mercury. Thanks to its nonexistent atmosphere and slow spin, the side of the planet facing away from the sun is extremely cold (-170°C) while the side facing the sun is extremely hot (420°C). A simple space suit isn’t going to cut it. Most of the suits designed for Mars or other space exploration are designed with insulation from cold in mind, not insulation from pure heat.

Could a Honda Pilot be insulated enough to withstand 420°C for 24 hours? If so I could be teleported to Mercury in my car. I stumbled across a website advertising thermal control paint for spacecraft and satellite applications, which has to have the smallest target audience of any marketing website I’ve ever seen. They report that their thermal control paint, AZW/LA-II, operates in temperatures ranging from -180°C to 1400°C, which is perfect for our Mercury use! On the website, they also describe the important components of an overall temperature control system, including radiator surfaces, heat pipes, and thermal isolation systems. This company seems like they know what they’re talking about. So, all I’d have to do is contract with them to get my Pilot outfitted with the necessary components of an interstellar heat reduction system, and I’d be good to go! I could sit in my space suit for oxygen and for protection from the near-vacuum atmospheric conditions, and hang tight.

My Pilot, complete with thermal shields, reflective coating, and a heat sink

Also, when reading the wikipedia page for Mercury I saw this:

Better start selling those stocks.

4. Venus

Now we’re getting really tough. Venus is the planet I mentioned in the introduction. As previously stated, Venus is very hot, albeit not as bad as Mercury. Surface temperatures are around 460°C, which is theoretically survivable in our heat-resistant Pilot. The problem(s) are twofold: the atmospheric pressure and the weather conditions.

Venus’ atmosphere is very dense—its pressure is about 93 times higher thatn Earth’s atmosphere at surface level. At these pressures, our current contraption will be instantly crushed. We might need to abandon the Pilot.

The Deepsea Challenger is a submersible designed (with notably fewer critics than Titan had) to withstand the pressure at the deepest parts of the ocean (depths of around 11km). According to Wikipedia, the pressure on Venus is equatable to the pressure of a water depth on Earth of about 1km. So, this submersible is well above the threshold for pressure survivability. As a bonus, it comes with air tanks, so I don’t need to worry about sitting in a space suit for the whole day. We could equip it with all the aforementioned heat shielding and be good.

The weather might be the bigger problem. According to the website for our thermal control coating, the coating is susceptible to degradation from radiation, atomic oxygen, thermal cycling, and micrometeoroid impacts. The first three aren’t really a problem on Venus but the last one is an issue. Any surface exposed to the Venus weather will be constantly buffeted by rocks and other debris thanks to wind speeds of up to 300km/h. Also, it rains acid. Venus is a lovely place.

One solution is to encase our contraption in a weather resistant dome shield. I have a cooler solution in mind.

A tunnel boring machine is a large machine used for quickly drilling large tunnels. They’re commonly used to excavate large amounts of earth for subway systems or traffic tunnels.

If we put one on the bottom of the Deepsea Challenger, could we dig ourselves into the ground quickly enough to escape the weather? Soil-tunneling machines can excavate at a rate of around 200m/week. This equates to 28.5m/day, and about 1.2 meters per hour. The dimensions of the submersible are 2.3m (7.7 ft.) X 1.7m (5.5 ft.) X 8.1m (26.6 ft.). If we laid it on its side, it would be dug underground in less than two hours with an industrial boring machine. Of course, the top would still be exposed, but we could just limit our weather shield to a disk at the top. I think this idea’s a winner.

5. Uranus

To no one’s surprise, the Gas Giants rank last. The final four planets have no defined surface, so it’s hard to determine what altitude I’d get teleported to. For each, I think it’s fair to teleport to the part of the atmosphere which corresponds to a pressure of about 1 Earth atmosphere. You may kvetch and say “well, that’s ignoring pressure! No fair!” and to that I say “wait and see how bad these planets get.” We’ll speed through these, as the contraption is very similar for each.

All of these planets will kill you. All feature astronomical wind speeds, massive storms, extreme radiation, huge temperature fluctuations, and no stable surface. These rankings are pretty much determined by how easy it would be to fly a plane in their atmosphere. In researching for this post, I found an XKCD that helpfully described the flight conditions for all the planets (there’s always a relevant XKCD). I’ll be agreeing with some of the rankings, but not all.

Uranus is the calmest of the bunch, relatively speaking. At 1 atm (atmosphere), wind speeds range from 360 to 180 km/h, which is very very fast. There is relatively little radiation compared to the other gas giants—about 100 times weaker than Jupiter, but we still need the lead walls. The gravity of Uranus is around .9g, so flying/floating is still difficult.

The Lockheed WC-130 is an American military plane designed to fly into the center of a hurricane. This will be our base plane for surviving the planets. We’ll redesign the walls to be lined with lead to block potential radiation rays, and add the thermal coating previously discussed. Here’s me on Uranus:

6. Saturn

Saturn is somehow orders of magnitude worse. The winds achieve speeds of up to 1,800 km/h, which is just absurd. We can try and super-reinforce our Lockheed, but it’s not looking good. Our only chance of survival is if we are teleported to a patch of the atmosphere where the storms are relatively tame. If we can fly with one long jet stream and not hit any vertical shear, we might be able to fly until we hit turbulence.

7. Neptune

Not looking too good so far. Neptune has the strongest winds (2,200 km/h) and largest storms of any planet in the solar system. The good news is that gravity is only about 1.1g, which means we can still theoretically fly a plane. The bad news is that we don’t know of any potential plane wing material strong enough to withstand these atmospheric conditions. Winds on Saturn are almost as fast as Neptunes, but Saturn’s storms and cloud patterns are much less volatile. Neptune storms have constantly-shifting wind patterns which make flying much more difficult. Our plane will get torn apart within minutes of trying to exist in such extreme storm conditions.

8. Jupiter

We’re dead. Not only does Jupiter have immense storms with lightning as strong as atomic bombs that strikes around four times per second in some thunderheads, but the gravity is 2.4g: our plane isn’t staying in the air. The only possible solution is to try and add massive jet engines to our Lockheed to try and offset the massive differential. Oh, did I also mention the constant bombardment of extremely lethal radiation? Teleporting to the atmosphere of Jupiter would result in our contraption being immediately shredded by unbelievably turbulent winds, blasted by giga-lightning, and rapidly pulled to crush depths, probably in that order.

Don’t mess with Jupiter.

Conclusion

And there you have it. A definitive ranking of how easy each planet would be to survive for 24 hours. My main takeaway from all of this? Space is scary. Good thing we have Mother Earth.

How to Write Greek Letters

Mon, 13 Oct 2025 00:00:00 +0000

What does every great statistician need the ability to do? If you answered easily explain complicated theorems or derive complex formulae by hand, you are incorrect. The true answer is that every stats person needs to be able to draw Greek letters on a whiteboard without them looking like scribbly blobs.

This is something I am unsurprisingly bad at. To help myself (and hopefully share this knowledge with others), I’ve compiled a list of Greek letters and how to draw them. With the hefty assistance of some LLMs, I created a rainbow sketchpad that shows the direction of a line through its color (red $\rightarrow$ purple). So, as you browse through the list of letters, you can tell in which direction the pen stroke should start when handwriting. You can also have fun experimenting with the sketchpad below:

The Greek Alphabet

If we get the term alphabet from the Greek system, do we really need to clarify that we’re using the Greek alphabet? Technically, we should be calling our system the “English Alphabet.”

Here’s a list of all the letters and how I write them (starting on red). As a disclaimer, these depictions are gotten from use in mathematics or statistics, not everyday handwriting. So, some of them might be needlessly complex or not follow convention.

Uppercase	Lowercase
Alpha	Alpha I used to draw it starting from the bottom, but starting from the top seems to give it a more full circle.
Beta	Beta The slant is important.
Gamma	Gamma
Delta	Delta
Epsilon	Epsilon
Zeta	Zeta I have no clue how to write this one, really. I don’t think I’ve ever used it.
Eta I think the slant makes it look more “Greek.”	Eta
Theta	Theta
Iota Tough one.	Iota
Kappa	Kappa I like to put a little tail at the top of the main stroke to distinguish it from a “k.”
Lambda	Lambda Half-Life reference.
Mu	Mu
Nu	Nu Again, the slight curve distinguishes it from a “v.”
Xi What in the world is this thing?	Xi The most feared of them all. Known far and wide by matrix algebraists. Not too bad if you think of it as a lowercase zeta with an extra loop.
Omicron Yippee! Easy.	Omicron
Pi Looks like a Torii.	Pi
Rho	Rho
Sigma	Sigma
Tau	Tau Another little curve to make it look more Greek than a “t.”
Upsilon	Upsilon
Phi Sometimes serifs are used at the top and bottom to distinguish it from the lowercase phi.	Phi
Chi	Chi
Psi The serif on the right side (and size of the symbol) differentiate it from the lowercase version.	Psi
Omega	Omega

And there you have it. Realistically, most of these aren’t very common in stats, but I figured it was a good idea to list them all anyway. It was definitely an adventure to try and write them all with my computer mouse instead of actually being able to draw them on paper, but I thought the rainbow line was too fun to pass up. Hopefully, some of these depictions will be a little useful to those of us who are still stats novices.

P.S. If you make any cool drawings with the sketchpad, send them to me!

Grad School Scramble III: Inter-rater Reliability

Sat, 23 Aug 2025 00:00:00 +0000

Well, here we are! Summer is over, the new students are flocking to campus, and I find myself in the beautiful and hot state of Arizona. Classes technically started on Thursday, but they mostly consisted of looking over syllabi and introducing yourself to your classmates. The ASU program is great so far: my fellow grad students are awesome, the faculty are trememdous, and I’ve already begun immersing myself in local communities like choir and card shops.

During one of my Friday classes, Professional Issues in Psychology (or, as it’s informally known, Intro to Being a Grad Student), the instructor Cheryl Conrad told an anecdotal story about inter-rater reliability. As she was telling it, I realized that I really don’t know much about the concept, and decided that it would be a good installment of Grad School Scramble. I have not been nearly as prolific with this series as I had hoped, but I wanted to finish one last post before school gets heavily underway. Who knows—maybe I’ll continue it into the semester.

Inter-rater reliability: Definitions and concepts

Psychological studies and tests can often require responses that are long, involved, or otherwise unable to be directly assessed numerically. When such responses are present in a test, they are often interpreted by an individual (or individuals) to convert them to a format that can be analyzed. For example, essays on AP exams are sent to independent readers that rate the essays on a scale from 0-6. In AP’s case, two independent readers will score each essay, so that an individual score isn’t biased by one single reader. For more general tests and studies, these readers are generally known as “raters,” and it’s critically important that they are accurately and reliably assessing each response.

How can we determine if raters are doing a good job? One method is by evaluating the inter-rater reliability: a measure of how much the raters are agreeing with each other. If they display a large amount of agreement, we have reason to believe that their judgments are valid; conversely, if they disagree frequently, there might be reason to doubt the validity of their ratings. With that said, how do we measure their agreement?

Before diving into agreement measures, it’s important to discuss what we even mean by “agreement.” When scoring is categorical, agreement is simple: the raters either agree or they don’t. When scoring is ordinal or better, however, the question of “scoring distance” comes into play. Raters who score an essay 5 and 6 respectively seem to agree more than raters who score an essay 1 and 6, but they still disagreed. How do we account for that? Furthermore, there are different types of rating. There exists the aforementioned scoring where raters are agreeing/disagreeing with a single rating, but what about if they’re both confirming an official score? What if they’re ranking different options? What if there are more than 2 raters? There are a variety of different measures to quantify agreement in each of these situations.

Simple probability of agreement

The first and easiest example of an agreement measure is the probability of agreement. This measure is only applicable for nominal data, but it was the standard method of determining inter-rater reliability for a lengthy era of quantitative methodology. When there are 2 or more raters scoring a response, it is easy to calculate the percentage of the time that they agree. We simply divide the number of identical scores by the total number of scores and obtain the percent agreement. When there are more than two raters, we can set up a matrix in which each row represents a variable of interest and each column is a rater score.

\[\begin{array}{c|ccc} VAR & Rater 1 & Rater 2 & Rater 3 & Agreement \\ 1 & 1 & 1 & 1 & 1.00 \\ 2 & 1 & 1 & 0 & 0.66 \\ 3 & 0 & 0 & 0 & 1.00 \\ 4 & 0 & 1 & 0 & 0.66 \\ & & & Total & 0.83 \end{array}\]

According to McHugh (2012) (who I shamelessly stole the matrix example from), the direct interpretation of the probability of agreement is the percent of valid data. That is, if you have 90% percent agreement, then 10% of the data in your study are erroneous.

Cohen’s kappa

As we can see from the previous example, the joint probability of agreement has some issues as a measure of inter-rater reliability. For one, it is only applicable for nominal data. Furthermore, its interpretation is overly simplistic and doesn’t tell you much about the direction of erroneous data. The biggest issue, however, is the one that statistician Jacob Cohen pointed out in the ’60s: it doesn’t account for agreement that arises due to chance. For example, if two professors were to rate student performance on a test as either pass/fail, and we asked them to give scores completely randomly, we would still expect them to coincidentally give the same rating about half of the time. On paper, they would seem to exhibit 50% congruence, but in reality there would be no systematic agreement between the two raters.

To account for this, Cohen developed Cohen’s kappa, stylized as $\kappa$, as a correlational coefficient of rater agreement. It ranges from $-1$ to $1$, with $-1$ representing perfect disagreement, $1$ representing perfect agreement, and $0$ representing the agreement we would expect due to random chance.

The formula is as follows:

$\kappa=\frac{p_o-p_e}{1-p_e}$

where $p_o$ is the observed percent agreement and $p_e$ is the expected percent agreement due to chance.

The intuition behind the formula is that $p_o-p_e$ represents subtracting the expected chance percentage of agreement from the observed percentage, leaving the “non-chance” percentage of agreement left. Dividing by $1-p_e$ rescales the result so that $1$ is perfect agreement and $0$ is random chance. For example, if you expect 50% of your agreement to be due to chance and your raters perfectly agree, then you would still get a kappa coefficient of $1=\frac{1-.5}{1-.5}$. If you expected 50% of your agreement to be due to chance and your raters agreed 50% of the time, then you would get a kappa coefficient of $0=\frac{.5-.5}{1-.5}$.

For 2 raters, $p_e$ can be calculated by a chi-square table where each class is a rater. For example:

\[\begin{array}{cccc} 100 & 12 \\ 3 & 90 \end{array}\]

If this is our chi-square table (forgive the poor $ \LaTeX $), and rater 1 is the columns (rating of pass/fail) while rater 2 is the rows (rating of pass/fail), then we can calculate the expected agreement due to chance as follows. We can find the sum total of each row and column as $112$ and $93$ for the rows, and $103$ and $102$ for the columns, respectively. The total $n$ of the ratings is 205. According to the rule of independent probability, if our data were all independent then knowing the row tells you nothing about which column some data might be in. Let’s take the example of the top-left cell, which is the intersection of row 1 and column 1. Formally, if $P(A)$ is the probability that a variable is in row 1 and $P(B)$ is the probability that a variable is in column 1, then $P(A\space and\space B)=P(A)\cdot P(B)$. By the definition of a chi-square table, $P(A)=\frac{100+12}{205}=0.55$ and $P(B)=\frac{100+3}{205}=0.50$. Thus, $P(A)\cdot P(B)=.28$. So, if all of our data were due to random chance we would expect the top-left-most cell to contain 28% of our data, or a total of about 57. We can repeat this process for the other cells and sum up the totals of the congruent cells to determine the percent agreement we would expect due to chance. At the end of this process, it turns out that this chi-square table results in a $p_e=.50$, which is an expected percentage of agreement due to chance around 50% of the time. (For more information on probability, feel free to read my blog post about basic concepts in probability)

Circling back to Cohen’s kappa: Once we know the expected probability $p_e$, we can finish our formula. Our total agreement $p_o$ is $\frac{190}{205}\approx.93$. When we substitute these numbers back into our kappa formula we get $\kappa=\frac{p_o-p_e}{1-p_e}=\frac{.93-.50}{1-.50}=0.85$.

Can anyone see an issue with Cohen’s kappa from this example? I have one: It makes the assumption that ratings due to chance would follow the distribution of rows and columns exactly equal to the numbers we observe. Due to this, Cohen’s kappa can sometimes be too critical of inter-rater reliability because it assumes that random chance ratings would look similar to the marginal proportions observed. It also assumes that raters are always independent. Despite this, $\kappa$ remains a more robust estimation of rater agreement than percent agreement.

Because $\kappa$ is a correlation coefficient, it cannot be directly interpreted.

Quick note: while I was researching and reading stuff for this post, I came across the aforementioned McHugh (2012) paper which stated that squaring kappa results in the “amount of accuracy” due to agreement between the raters. This is a foundational article, with almost 22,000 citations. Regardless, I could not find any information to back this idea up, and plugging some example numbers into a quick Cohen’s kappa calculator seemed to disprove it. Their justification was that squaring any correlation coefficient results in an estimate of the variance explained, but I am not knowledgeable enough to confirm nor deny this. Can anyone smarter than me chime in and help with this?

Regardless of the interpretation of kappa squared, we have accurate and verifiable standards for Cohen’s kappa. The following table shows which values are deemed acceptable:

\[\begin{array}{c|ccc} 0.00-0.20 & Slight\space agreement \\ 0.21 - 0.40 & Fair\space agreement \\ 0.41 - 0.60 & Moderate\space agreement \\ 0.61 - 0.80 & Substantial\space agreement \\ 0.81 - 1.00 & Almost\space perfect\space agreement \end{array}\]

Weighted Cohen’s kappa

In situations where raters need to make ordinal scorings with more than 2 categories—such as ranking patient pain from mild/moderate/severe—Cohen’s kappa has another issue: It does not differentiate between “levels” of disagreement. In the example of patient pain, we might want to quantify the disagreement between a doctor recording “mild pain” and a second doctor recording “severe pain” as a more extreme disagreement than “mild pain” and “moderate pain.” However, if we set up our 3x3 contingency table and calculate the expected percentages of agreement/disagreement, all disagreements are weighted equally.

To solve this problem, in 1968 Cohen published an extension to kappa known as weighted kappa (although, arguably, normal kappa is a special extension of weighted kappa). Using weighted kappa allows for a researcher to assign weights to disagreements such that some can be viewed as more extreme than others.

How does it work? Well, the formula for weighted kappa is as follows:

$\kappa _w{}=1-\frac{\Sigma v_{ij}p_{oij}}{\Sigma v_{ij}p_{eij}}$

To obtain this formula, Cohen began with some simple algebra from the formula for $\kappa$. As a reminder,

$\kappa=\frac{p_o-p_e}{1-p_e}$.

If we let $q$ represent the proportion of disagreement and define $q=1-p$, then we can do the following algebra:
$q+p=1$,
$p=1-q$.

From here, we can represent our observed agreement and expected agreement in terms of $q$:
$p_o=1-q_o$,
$p_e=1-q_e$.

Now, we can substitute these values into the original formula for $\kappa$:

$\kappa=\frac{p_o-p_e}{1-p_e}=\frac{(1-q_0)-(1-q_e)}{1-(1-q_e)}$.

After some simple algebra, we get $\kappa=\frac{q_e-q_o}{q_e}$. If we then divide both sides of the fraction by $q_e$, we get $\frac{1-\frac{q_o}{q_e}}{1}$, which quickly simplifies to:

$\kappa=1-\frac{q_o}{q_e}$.

Now we have our original kappa formula in terms of the proportion of disagreement between the raters. Cohen next introduces a weighting concept based on the strength of disagreements. If we imagine a contingency table set up as so:

Then we want to weight the disagreements so that more severe disagreements are more costly to $\kappa$. The way in which this is done is up to the researcher; but, as Cohen stresses, it must be done prior to the collection of data. One common weighting method is linear weighting, where the weight assigned to each disagreement is simply the distance between the two cells:

The weights are the numbers in purple

Another alternative is to use quadratic weighting, which is when the distance between the incongruent cells is squared. Finally, as can be shown algebraically, when all disagreeing cells are weighted equally it recovers the original definition of $\kappa$.

$\kappa{_w}$ simply represents the original kappa formula with $p_o$ and $p_e$ replaced by weighted proportions of disagreement. So, for each cell, we multiply the proportion (or frequency) by the weight, and get our weighted proportion. When we add this weighting factor back into our modified kappa formula we get the final $\kappa{_w}$ formula, which is:

$\kappa _w=1-\frac{\Sigma v_{ij}p_{oij}}{\Sigma v_{ij}p_{eij}}$

where $v_{ij}$ represents the weighting factor for cell $i,j$.

Finally, we want to prove that this recovers the original kappa formula when all weights are equal. If we assume that all cells are weighted equally, then $v_{ij}$ becomes some non-zero constant, which can then be pulled out of the summation. After that, the constant can be cancelled out of the numerator and denominator, giving the formula:

$\kappa _w=1-\frac{p_{oij}}{p_{eij}}$,

which can be understood as our original kappa proportion of disagreement.

For the purposes of space and time, I won’t be going through a full hand-calculated example of weighted kappa, but I hope that this run-through of the formulae was enough to serve as a decent introduction.

Pearson’s r

If ratings are continuous, then Pearson’s r can be used pairwise as a measure of inter-rater reliability. However, this is mostly a poor choice, because it fails to take into account systematic biases between the two raters. For example, if one rater scores a measure (1, 2, 3, 4, 5) while another rater scores pairwise the same measure (2, 3, 4, 5, 6), then their correlation will be 1, but there will be difference in their agreement. So, I’m not going to go very in-depth into the usage of r as a measure of inter-rater reliability. Just know that it’s technically an option.

Intraclass Correlation Coefficient

So, what should we use if ratings are continuous (interval or ratio)? One option is to use the Intraclass Correlation Coefficient (ICC). From a mixed-models standpoint (as was explained to me by the amazing Catherine Bain), the ICC is the amount of variance of the intercepts divided by the total variance (which comprises the variance of the intercepts + the unwanted variance). So, if the unwanted variance is higher than the variance of the intercepts, then the ICC is lower than $.5$ (which is a poor result). On the other hand, if the variance of the intercepts composes a large amount of the variance than the ICC will be higher.

We can extend this idea to inter-rater reliability by considering the items as the clusters and determining how much variance in score is due to item difference vs. rater difference. If we look at the previous chart, if we let “Group Number” represent the different items and the blue dots represent the scores that raters assign to the item, then we can observe how the individual item is much more predictive of a score than the distinctive rater. This is what we want to see—and displays a good ICC of $.91$. Conversely, if there was more of a difference between raters than between items, then there would be more “unwanted variance” than intercept variance, and the ICC would be much lower.

Unlike Cohen’s kappa, the ICC compares to a theoretical null model of “no intercept variance” which is more straightforward than the kappa assumption about what chance agreement would look like. This can make it more useful under certain conditions. However, according to Wikipedia the ICC does make the assumption that raters are exchangable for a certain item. In other words, there is no meaningful way to order the raters. This assumption is violated if there is systematic bias between different raters (which would result in some raters being naturally more critical than others), which is likely in real-world scenarios, so debate exists about whether other measures should still be used in place of the ICC.

As always, I am not a professional, so I feel the need to give the disclaimer that I don’t have a full mathematical understanding of how the ICC works. Nor do I have an understanding of mixed models past what you can get from a couple of YouTube videos. Still, I hope that most of the information presented is at least somewhat accurate.

Concluding thoughts

Well, there you have it. The results of 3 days of reading information about inter-rater reliability. Is it super, incredibly useful? I’m not sure—I’ve never been a part of a study with multiple raters. In any case, it certainly was interesting. One of my big take-aways is how amazing Jacob Cohen’s work was, especially for the time. I learned about kappa before investigating weighted kappa, and the weighted version seemed a very powerful yet natural extension of the previous work. I tried for almost an hour to sit down and derive the formlua for weighted kappa from the unweighted version, and couldn’t figure it out (even with the weighted formula sitting in front of me). Lo and behold, Cohen does it in like three lines on the third page of his paper. Sometimes it almost feels like we’re falling off the shoulders of giants.

Grimmsnarl to the Rescue

Tue, 17 Jun 2025 00:00:00 +0000

Aaaand we’re back! I am proud to announce that I recently competed at the 2025 Pokémon North America International Championships (NAIC), placing 136th out of 3,812 competitors. This tournament took place in New Orleans, LA, and the whole trip was a delight. I traveled down with a few friends and my partner, and we got to see Bourbon Street and the French Quarter. However, that’s not really what we’re concerned with. Time to run back the tournament and see how I did.

Deck Choice and Meta Calls

For this tournament, I chose to run a new deck: Grimmsnarl Froslass. NAIC was a tournament with some relatively special circumstances surrounding it: a new set of Pokémon cards released the same day as the tournament. This meant that there was an influx of potential new decks and strategies, and I wanted to bring one of the new decks in the hope that my opponents wouldn’t be extremely prepared for the matchup. Unfortunately, I made this decision about three weeks prior to the tournament, and in the days leading up to NAIC the popularity of Grimmsnarl exploded. The whole community was talking about it as a top meta contender, and everyone was preparing/teching for it. Still, just because a deck is well-known doesn’t mean it’s a bad choice, and I certainly believed in the power of the deck. Here’s the list I cooked up:

Notice the Tatsugiri

Grimmsnarl is a slow deck: a stark contrast to the Gholdengo deck I brought to Milwaukee. At my last regionals, I wanted a quick deck to be able to finish all 3 games in a 50-minute match. Why did I make the switch? Well, one of the big things that annoyed me about Gholdengo was the matchup into 2-2-2 decks like Raging Bolt. Aggressive decks like these want to KO big 2-prize Pokémon each turn as fast as possible to close the game out in just 3 attacks. If two decks like these are facing each other, falling behind in the prize race is a death sentence that is almost impossible to comeback from. (Side note: the Gholdengo-Bolt matchup is indeed a lot more nuanced, with Gholdengo having the opportunity to present a single-prize board state, but Bolt being able to counter that by only playing down one 2-prizer to enable noctowls and setting up a baby bolt to even the trade with a single-prize board state of their own. The prize map overall favors Bolt when both sides are playing optimally. Still, the point stands that falling behind in the map means you lose). Slow, set-up decks like Grimmsnarl have many more comeback mechanics, and as a result provide you many more opportunities to outplay your opponent. As I was gaining confidence as a player going into this tournament, I wanted to bring a deck that would allow me to consistently outplay my opponent.

Also, I was a big believer in Grimmsnarl’s matchup spread. I believed that it was favored into both Dragapult and Gardevoir: the two decks that had defined the meta in the months leading up to this tournament. I also believed that it had an OK-ish Bolt matchup, which was projected to be a popular deck as well. The only matchup I was worried about was Joltik Box, but I didn’t think I’d see too many of them. Spoiler alert: I did.

The overall strategy of the deck is to utilize Grimmsnarl ex alongside Froslass and Munkidori to spread massive amounts of damage on your opponent’s board. Alongside cards like TM: Devo, you can take huge swing turns with multiple KO’s to win games.

Let’s take a look at my matches and see how I did.

Round 1: Win against Poison Roaring Moon

My first round went swimmingly. My opponent was running a deck centered around Roaring Moon ex with a poison package intended to “donk” the other player. In the Pokémon TCG, if your active Pokémon is knocked out and you don’t have any other benched Pokémon to replace it with, you lose. With the help of cards like baby Pecharunt and Perilous Jungle, the deck aims to take out the opponent’s starting Pokémon before they can even play a single card.

The deck’s main attacker if the donk doesn’t work

Fortunately, my Impidimps are immune to the effect of Perilous Jungle, and as a result are unable to be donked. Unfortunately, I cold-started Snorunt on my first game. All I could do was sit and watch as my opponent attempted to piece together the donk combo. He missed it by a single nest ball, and I was luckily able to take down the 1st game after he whiffed a supporter on the next turn. The second game went well - I started an Impidimp in the active and a Snorunt on the bench. He punished my misplay of benching Snorunt when he immediately whipped out the donk combo + an Iron Bundle to KO my Snorunt. However, using a Defiance Band and some residual poison damage on Impidimps, I was able to target down his Roaring Moon ex’s with energy attached and prevent him from ever pulling off a Frenzied Gouging. 1-0-0.

Round 2: Win against Raging Bolt

This round went well too. I was a bit nervous to see my opponent flip over a Teal Mask Ogerpon; Bolt is one of the worse matchups for Grimmsnarl. They can always oneshot a Grimmsnarl with the Ogerpon and threaten huge amounts of pressure with the baby Raging Bolt. However, this is the match that made me incredibly thankful I had chosen to run Maractus:

Leading up to the tournament, there was a huge meta discussion surrounding Maractus. Its attack prevents the opposing Pokémon from retreating, which is an incredibly powerful effect against decks that don’t run some sort of switching tech. A few weeks ago, a Dragapult list running Maractus won a big regional, so people started preparing for Maractus by putting switch cards in their decks. Due to this, many people chose to cut Maractus, as they felt that it was now redundant. I viewed the card a bit differently: not as a “lock one Pokémon and you win” card but rather as a tool to slowly run your opponent out of resources. Even though my opponent ran 2 switch effects, I still was able to force my opponent to use his resources suboptimally and buy enough time for my Froslass + Munkidoris to build up massive damage on his board. 2-0-0.

Round 3: Win against Dragapult Dusknoir

My opponent for this round was a really cool guy—very friendly and we had a nice chat before the match started. He was on Dragapult Dusknoir, one of the top decks in the format. I felt confident in this matchup, and I took game 1 after a long back-and-forth. He played it very well, but he prized what I believe was his only Dusknoir, which really hurt his gameplan. I had a cool play where I swung into his Budew with an Impidimp for 10 to set up a 20 damage Munkidori KO the following turn, but other than that the game was uneventful. Our first game took a long time, and so we only had around 20 minutes left in the round. We ended up not finishing our second game after I Iono’ed him low and he was unable to escape Maractus on one pivotal turn. 3-0-0.

Round 4: Loss against Grimmsnarl Froslass

This was one of the best matches of the tournament. My opponent was a very skilled player who placed in the top 64 of multiple regionals this season. He was on the same deck I was playing: Grimmsnarl Froslass. Our first game was an absolute slugfest that took around 45 minutes. At the very beginning of the game, I was tactically forced to Professor’s Research away my only Super Rod, which I knew would come back to bite me. The other thing that hurt is that he was playing four Munkidori as opposed to my three.

In the mirror match, whoever establishes the most Munkidoris will usually win. The ability to reflect huge amounts of damage back over to your opponent is incredibly powerful in this matchup, and the best strategy is normally to try and kill your opponents Munkidoris instead of their Grimmsnarls. I was able to maneuver him into a position where he had to retreat two energy off of his only Grimmsnarl in play, leaving him a few turns later with a board of only a no-energy Grimmsnarl and four Munkidoris. Unfortunately, I had only one Munkidori in play myself, and I was unable to use Super Rod to get any back. This left me in a position where if I were to swing into his Grimmsnarl, he would be able to move 120 damage back onto my Grimmsnarl, making it so that I couldn’t even KO his Grimmsnarl next turn and setting me up for a loss 2 turns later (prizes were 2-2). This put us in a situation where both of us were passing back-and-forth because swinging for any damage would be a suboptimal play.

What my opponent eventually realized is that he was going to deck out before I was, so he had to take the risk of putting a Snorunt down to try and establish Froslass. I was able to Iono, Counter Catcher up a Munkidori, and swing into it putting 30 on the Snorunt, with the logic that if he whiffed establishing the Froslass next turn I would immediately be able to do 30 more to the Snorunt and win the game. Unfortunately, he drew into the only Froslass he had left in deck, which was around a 3/15 chance. Because of this, I unfortunately lost the game on the following turn when he drew into an energy as well to attack with Grimmsnarl and win.

With only 4 minutes left in the match, there’s no way I can manage a win in game 2: a loss for me. Even though I lost, I thought this was an incredibly strategic match, and I was happy with how I played. 3-1-0.

Round 5: Loss against Gardevoir

In contrast to the previous round, this was one of the most frustrating moments of the tournament. My opponent was on Gardevoir: a matchup that was theoretically favored for me. Because of this, I was a bit overconfident and complacent during our match, and a few suboptimal plays in game 1 put me in a losing position. I tried my absolute hardest to squeak out a win, but my opponent was victorious. There was only 25 minutes left in the round, and I didn’t want to settle for a draw in a favored matchup. So, I tried to turbo through the next game and take a strategically weaker line for the sake of trying to end the game early, and was punished with an 0-2 for the round. 3-2-0.

Before my next round, I go into Zen Mode. I sit in a bathroom stall for a few minutes and listen to dubstep while hyping myself up. I have to lock in—if I approach my games with the misguided and foolhardy confidence I entered this round with, I will certainly lose. My next rounds will have to be played to perfection. I check my phone: pairings are up.

Round 6: SPOLER BLOCKER against Joltik Box

Well, this is horrifying. Round 6 of 9 and I’m already super close to elimination. Any record below 6-2-1 (6 wins, 2 losses, 1 tie) means you can’t make day 2, so I was in a position where I had to win rounds 6, 7, and 8 (and potentially 9 if my opponent didn’t want to intentionally tie) to make day 2. I’m riding the lightning.

My next opponent is quite good: I checked his Limitless page before the round and he has multiple Top 32’s at major regional tournaments. To my absolute horror, he chooses second and starts Miriadon. Looks like we’ve got a Joltik Box.

As I previously mentioned, this matchup is atrocious for Grimmsnarl. They run Iron Hands to take 2 prizes on your small Pokémon if you play it slow, and run Iron Leaves to get around Maractus, pivot energy off the Iron Hands, and oneshot your Grimmsnarl.

I quickly lose the first game, and my dreams of making day 2 are flashing before my eyes. Game 2 goes more in my favor: I am able to choose to go 2nd, which severely hinders Joltik Box’s ability to turbo out an Iron Hands. Due to some solid disruption and a quickly established Froslass, my opponent scoops up the game fairly quickly. Game 3 gets off to a horrible start: I have Munkidori in the active and no way to search out any other Pokémon. I attach an energy and pass. My opponent immediately starts turboing out cards, playing Miriadon, Lillie’s Clefairy, and two Iron Hands. He benches a Mew ex, and then Crispin’s to the Lillie’s Clefairy on the bench. He retreats the Joltik in the active, promotes the Lillie’s Clefairy, and announces Full Moon Rondo for 120 damage, KOing my Munkidori and donking me to win the game. It’s over: I’m out of the tournament.

But wait—something’s wrong. How did he retreat the Joltik? It has a retreat cost of 1. I ask him to clarify how he retreated the Joltik, and he looks at his board for a few seconds, looks back up and me, and goes “You know, that’s a really good point.” Apparently he missed something, or didn’t consider the retreat cost, or was mentally operating as if he already had a Latias ex in play. Whatever the case, this is an absolutely egregious misplay against Grimmsnarl. Because he was trying to max out the damage of Full Moon Rondo (which does more damage the more Pokémon you have in play), he had bench-locked himself with two Iron Hands and no Latias ex. This is a death sentence. I am able to prevent him from ever attacking with Joltik by stalling on one Iron Hands and switching to stall on the other once the first one was almost powered up. I don’t ever attack until I have a fully set-up board, and he quickly loses the 3rd game.

I had mixed emotions about this match. My opponent was a cool guy: he was certainly not trying to pull a quick one on me and seeing such a bad misplay occur is unfortunate. On the other hand, I played perfectly to take advantage of the misplay, and avoiding mental lapses is a crucial part of competing in tournaments of these stakes. At the end of the day, a win is a win. Onto the next round 4-2-0.

Round 7: Win against Joltik Box

This round, I feel good. My opponent again starts Miriadon and chooses to go second, but this time I am filled with the confidence of just having beaten a Joltik Box. If I did it once, I can do it again.

I quickly take game 1 after my opponent gets off to a slow start. I am able to rare candy Grimmsnarl on turn 2 and prevent him from ever swinging with a Joltik. Game 2 is going my way again, when my opponent’s nerves get the better of him and he makes a massive mistake: he accidentally shuffles his hand back into his deck when performing some other action like a Nest Ball. I was already on pace to win that game, but that mistake was the final nail in the coffin. My opponent scoops it up, and I take the victory. Despite such a big error, my opponent seems very happy with how he did in the tournament, and wishes me luck.

5-2-0.

Round 8: Win against Dragapult Dusknoir

I’m in shock heading into the next round. My previous two rounds (both of which I had to win) were against my worst matchup, and somehow I’m surviving. Both of the wins felt like gifts: they were due to some huge, avoidable error made by my opponents. Nervously, I check my pairing for round 8. I’m against the guy who got 10th at the Mérida Regional Championships and 5th at the Special Event Bogotá. No more free wins against this guy.

He’s playing Dragapult Dusknoir, much like my round 3 opponent. Our first game is a doozy: I am able to take a little bit of a lead by KO’ing his Budew while he’s still under itemlock from me. Unfortunately, he is later able to piece together the perfect Dragapult turn and wipe out all of my Impidimps while establishing a Dragapult. I try and slow him down by TM: Devo’ing his rare candy’d pult (the only one with energy on it), but he top decks a second rare candy to get right back in the game (he even showed me and we chuckled about it). Eventually, he gets down to 2 prizes with two powered-up Dragapults on the board. I am down to 4 prizes thanks to his Dusknoirs, but I’m in bad shape. My only play is to Counter Catcher up his Fezandipiti ex and stall on it with Maractus. I do this for two turns, and somehow it seems as it he has no way of escaping it. He uses Arven but doesn’t grab a switch, and even benches a Duskull. Once I see that he’s trying to build up a Dusknoir to KO my Maractus and escape the lock, I know I am still very much in this game. I start building back up my board, and eventually establish a Froslass. I piece together a four-turn plan which should hopefully end with me KO’ing his Fez and a benched Drakloak on the same turn, and he’s attempting to build up a Dusknoir to escape the lock. The pinnacle of my strategy occurs when he KO’s my Maractus, freeing his Fez the next turn, but he goes down to 1 prize card. I iono him, Counter Catcher up the only Pokémon (Drakloak) he has in play with no energy, and hope that he whiffs an energy to retreat. He digs with his recon directive, but fails to get an energy, and passes it back over to me. I’m able to perfectly move damage onto his Fez and swing into his Drakloak with Budew to take my last 3 prizes.

Our second game, my deck runs very well, and I’m able to spike Secret Box off of my prize cards at the perfect moment to secure a win. 6-2-0.

Round 9: Tie against Dragapult Dusknoir

If I tie this round, I go 6-2-1 and make day 2. The issue with offering a draw (mutually confirming a spot in day 2) is that a tie in this round means that my opponent and I are likely out of the running for top cut. Still, I don’t mind. My goal was to day 2, and I will be super happy with a tie.

As I walk to the table, I see my opponent already has his hand extended, and I know I’ve done it. We shake hands, sign the match slip as a tie, and congratulate each other on making day 2 at the hardest and largest Pokémon tournament of the entire year. It has been DONE.

Round 10: Tie against Joltik Box

Honestly, there’s not much to say about my Day 2 matches. I’m on a victory lap and feeling good. Not even the fact that I’m going first against a Joltik Box can dampen my spirits. I actually manage to take game 1, but lose game 2 due to some minor misplays I could have avoided. We don’t have enough time to complete game 3, so I start the day off with a tie. 6-2-2.

Round 11: Tie against Raging Bolt

Another tie. My opponent gets an extremely powerful start during our first game, and blows me off the board. I manage to fend off the aggression during our second game, and take it as time is called. 6-2-3.

Round 12: Win against Raging Bolt

At this point, I’m getting a bit disappointed. Games against Joltik Box and Raging Bolt are not only unfavorable matchups, but they’re also not very fun. Both decks are designed to simply apply maximum pressure as quickly as possible, so a lot of strategic maneuvering (like of the kind I would get against other slow decks) isn’t present. My opponent is also playing a list absolutely tech’d to the max against Grimmsnarl: He has Switch, Professor Turo’s Scenario, and Picnic Basket.

Still, he makes a few minor overbenches and I am able to capitalize on his mistakes and take down both of our games.

Round 13: Win against Eevee Box

Round 13 was an absolutely fantastic round to finish the tournament on. My opponent was a great guy, and we were chit-chatting back and forth throughout the game. I totally put my foot in my mouth by asking him to speed up his actions and then immediately taking a 5-minute turn myself, but he just poked fun at me and we laughed it off. I was able to win the first game thanks to Budew, taking down all of his Eevees on one pivotal turn.

The final moments

Our second game was a close one, and it ended up going to time. It came down to a situation where he had no Hoothoots in play, and I was able to stall his Latias ex with Maractus and Iono him to 1. He told me that he had a switch in deck, and was able to dig 2 cards with Mew ex. Multiple judges were watching, so we went into full theatric mode and flipped both of the cards on the table: no switch. I take down a fun albeit slightly undeserved final round. GGs Donny.

Conclusion

Overall, I placed 136th / 3,812 participants. Damn, I kinda crushed it. I was super, super happy with how I did, and felt very confident about my deck choice and preparation. There were a few minor details in the Grimmsnarl list that the top players changed for NAIC, but I was happy with my overall list and tech choices. I believe wholeheartedly that Maractus was the correct play, alongside the Tatsugiri that I was so confident in. I was also very proud of my play. I was able to recognize minor errors (such as those in rounds 5 and 10), and I believe that recognizing the error is the important thing. The big thing, for me, is that I made day 2: something that many professional players failed to do at this tournament.

What will happen to my Pokémon hobby once I start grad school? I’m not completely sure. NAIC was, for me, the culmination of all the play and preparation that has been occurring since I picked up the hobby back in October. I wanted to achieve a high placement at this tournament, and I was successful. I’m sure I’ll keep playing, but I probably won’t be able to commit as much time as I have been these past few months. Still, as long as I’m a part of the Pokémon community, I will be able to brag about the time I placed in the top 3% at NAIC.

Grad School Scramble II: Much Ado About Probability

Mon, 09 Jun 2025 00:00:00 +0000

Recently, I woke up to a phone call from my grandfather asking me to explain the Monty Hall Problem in simple terms. For those of you who are unfamiliar, this problem is famous in the field of probability for confounding even highly-training individuals (like mathematicians and academics) due to its unintuitive nature.

The problem is thus: Imagine that you’re on a TV show, and the host asks you to pick randomly from three doors. Behind two of the doors is a goat, and behind one of the doors is a brand-spanking-new car. After you select one of the doors, the host then intentionally selects one of the other doors to reveal a goat. He then asks if you want to switch your guess to the other door or stay on your current guess. Is it to your advantage to switch?

Many people say that it doesn’t matter: your choice is a 50/50 between your door having a goat or a car. However, the true solution is that it is definitely to your advantage to switch! Swapping doors gives you a 2/3 chance to win the car.

The reason for this has a lot to do with probability theory and other unintuitive math, but it boils down to the idea that if you started off picking a goat, you’ll win the car by swapping after the host reveals the other goat. Only if you start on the car would swapping win you a goat. You can also consider the three different possibilities in the chart below:

As we can see, 2/3 scenarios win the car by swapping.

After (unsuccessfully) trying to explain this to my grandfather and other family, it got me thinking about the topic of probability in general. I have actually never taken a probability course, and it’s one of the areas of quantitative methodology that I’m probably weakest in. So, I figured that it would be a good idea to get a basic intro to some of the ideas in probability theory. Thus the second installment of Grad School Scramble™ was born.

Preparation

To see what topics I didn’t already have a good grasp of, I took an open-source final from an MIT probability course and graded myself using the provided key. Holy smokes, I have a long way to go. I had a solid understanding of most of the counting/set theory stuff just because I have taken a course on discrete mathematics, but most of the other material I was lost on. After my self-confidence was ground into pieces, I realized that a blog post covering an entire probability course was probably unfeasible, so I came up with a list of easier concepts to learn (and saved the multivariate calculus stuff for another time).

Conditional Probability

Conditional probability was the first concept that I didn’t have much prior experience with. I had worked with various concepts related to probability and counting in my discrete math course, including the principle of inclusion/exclusion and different methods of counting with combination/permutation rules. However, we never explicitly forayed into the world of conditional probability.

At its core, the idea of conditional probability answers the question “how does the likelihood of an event change given additional information?” When I was first reading about it, my intuition told me that you could answer this question with simple counting techniques: for example, what is the probability of a card from a standard deck having a suit of hearts given that its color is red? It seemed as if you could simply use logic to determine the sample space (red heart, red diamond, black club, black spade) and only use the subset containing the necessary event. However, this method is a bit constrained and only applicable for simple problems.

A formal definition of conditional probability is as follows:

$P(A\mid B)=\frac{P(A\cap B)}{P(B)}$

This reads as “the probability of $A$ given $B$ is equal to the probability of $A$ and $B$ divided by the probability of $B$.”

I had never seen this formula before. Intuitively, it makes sense, though. If $A$ and $B$ are dependent events, then dividing out the probability of $B$ from the intersection of $A$ and $B$ seems to leave you with only the probability of $A$ (given that $B$ has already occurred).

Another way to interpret this is by considering venn diagrams.

When we state that $B$ has already occurred, we can discard the entire region outside of $B$ (everything in pink). Thus, we are left with $\frac{P(A\cap B)}{P(B)}$ as the total probability of $A$ occurring.

One important consideration to make is that we can rewrite the formula as $P(A\mid B)P(B)=P(A\cap B)$ by simple multiplication. Thus, if we know the conditional probability of something we can use this formula to find the probability of event intersections.

Let’s use this formula to solve a problem. Approximately 10% of cars are red. What is the probability that two red cars get into an accident on the highway given that one of the cars is red? We can define $P(A)$ as the probability that the crash has two red cars and $P(B)$ as the probability that at least one car is red. Because $A$ is necessarily a subset of $B$ (for both cars to be red, at least one must be red), we can see that $P(A\cap B)=A=.01$. $P(B)$ is the complement of neither car being red, so $P(B)=1-(.9\cdot.9)=.19$. Because $\frac{.01}{.19}\approx.053$, the overall probability $P(A\mid B)\approx.053$.

It seems a bit counterintuitive that the answer isn’t just $.1$. After all, it seems like we know one of the cars is red, so the probability of the other car being red as well should just be $.1$. However, we don’t specifically know which car is red. Imagine that we specify the cars as Car 1 and Car 2. Our event space is as follows (for $R=red$ and $N=not red$): ${(R,N), (N,R), (R,R), (N,N)}$. When we use the logic of “one of the cars is red, so the probability of the other one being red as well should just be $.1$”, we implicitly make the assumption that we’re talking about Car 1 being the red one. However, the event that at least one of the cars is red comprises the set ${(R,N), (N,R), (R,R)}$ while the event that Car 1 is red comprises ${(R,N),(R,R)}$. Because $(N,R)$ is non-empty (we know that it could be the case that Car 2 is the red one), we know that the probability of both cars being red is smaller when we only know one car is red compared to when we know Car 1 is red (because in the first scenario $(R,R)$ is selected from a larger event space than in the second).

Another note: Two events can be considered formally independent if $P(A\mid B)=P(A)$. This is because they are independent if knowing that $B$ has occurred does not change the probability of $A$. It then follows that if two events are independent then $P(A\cap B)=P(A\mid B)\cdot P(B)=P(A)\cdot P(B)$, which follows our prior understanding of probability.

Bayes’ Theorem

Throughout the murky, dark waters of statistics and probability, one beast lies dormant, slumbering, waiting for an unfortunate soul to stumble across its gaping maw. This beast is none other than Bayes’ Theorem, striking fear into the hearts of frequentists everywhere (before you write an angry email—I know that Bayes’ Theorem itself is not wholly relegated to Bayesian statistics).

Much like conditional probability, I hadn’t much experience with Bayes’ Theorem before looking into it. In one of my undergraduate psych classes with Dr. Terry, we discussed the famous Kahneman and Tversky base rate fallacy in terms of Bayes’ Theorem: where individuals fail to incorporate the base rate of an event and obtain incorrect probabilistic deductions. Here is the theorem in all its glory:

$P(A\mid B)=\frac{P(B\mid A)\cdot P(A)}{P(B)}$.

This theorem lets us find the inverse of a conditional probability: If we know the probability of $A$ given $B$, we can now find the probability of $B$ given $A$. The proof follows from our previous conditional probability formula:

$P(A\cap B)$ is symmetric with respect to $A$ and $B$, so

$P(B\mid A)\cdot P(A)=P(A\cap B)=P(A\mid B)\cdot P(B)$.

Then, if we divide by $P(A)$ we obtain Bayes’ Theorem.

We can use this tool to answer the Monty Hall problem discussed earlier. If event $A$ is that the car is behind our door (door 1), and event $B$ is that the host opened door 3, we find the probabilities and substitute them in:

$P(B\mid A)$ will be $.5$ because the probability of the host opening door 3 given that the car is behind our door is $.5$ (he could have chosen either door 2 or door 3).

$P(A)=\frac{1}{3}$

$P(B)=.5$ (There are three scenarios: the car is behind our door, which gives Monty a .5 probability of picking door 3, the car is behind door 2, which gives Monty a 1 probability of picking door 3, and the car is behind door 3, which gives Monty a 0 probability of picking door 3. Thus, the total probability [$(.5 + 1 + 0) * \frac{1}{3}$] of Monty picking door 3 is .5)

So:

$P(A\mid B)=\frac{.5 \cdot \frac{1}{3}}{.5}=\frac{1}{3}$.

In other words, the probability of the car being behind our door given that Monty opened door 3 is $\frac{1}{3}$. Thus, it is in our interest to switch doors.

Bayes’ Theorem can also be stated as $Posterior=Prior \cdot Likelihood$, where each part of the statement is part of our original formula.

Common (Discrete) Probability Distributions

Next, I’d like to cover a few distributions that I was a little shaky on. It’s easy to fixate on the normal distribution as the “big one” of important distributions, but many data-generating processes are non-normal. Also, this gave me a chance to practice my plot-making skills in R.

For discrete distributions (which all of the following distributions are), the probability mass function, or pmf, is the function which gives the probability of a random variable from the distribution being equal to a given variable. This function is written $p(a)$ where $a$ is the variable of interest. This will be important as we examine different distributions.

Bernoulli Distribution

The Bernoulli distribution is a simple one: it models one trial in an event with a binary outcome. So, for example, if we’re observationally assessing the behavior of dogs being walked in a park (and want to determine whether or not they will urinate on a fire hydrant), the behavior of one specific dog would be modeled by a Bernoulli distribution.

The Bernoulli distribution takes one parameter, $p$, which represents the underlying probability of a $1$ for the outcome. If 70% of dogs will pee on the fire hydrant, then $p=.7$. If $X$ is the variable of interest, by definition $P(X=1)=p$.

Here’s an example of a Bernoulli distribution with $p=.7$:

Binomial Distribution

The binomial distribution is an extension of the Bernoulli distribution. It models the probability of an event (binary, represented by a Bernoulli distribution) occurring a certain number of times in a set number of trials. It has two parameters: $n$ (the number of trials) and $p$ (the probability of a success).

For example, if we wanted to determine the probability of flipping a certain number of heads in ten total flips of a coin, we might use the following binomial distribution:

By looking at this distribution, we can see the approximate probability of flipping a certain number of heads out of 10 flips with a fair coin ($p=.5$). For example, the chance of flipping exactly 3 heads is around $12\%$. It looks Gaussian, but this is only because our probability of a heads is $.5$. If we want to imagine that the coin we are flipping is unfair, we can assess the following distributions with $p=.2$ and $p=.9$, respectively:

How do we obtain these distributions? The pmf for a binomial distribution with probability $p$ and # of trials $n$ is defined as

$P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}$

From this, we can see why it’s called the binomial distribution. The $n$ choose $k$ part of the formula is a binomial coefficient: coefficients of binomial expansion that follow Pascal’s Triangle.

Geometric Distribution

Geometric distributions model the probability of a certain number of failures before a success in a sequence of Bernoulli trials. In other words, they show how many tails you can expect to flip before you get a heads.

If $X$ is the number of tails, the pmf for a geometric distribution is defined as:

$P(X=k)=(1-p)^kp$

where $p$ is the probability of a success. The intuition behind this formula is that it’s simply a representation of the odds of getting the desired sequence. For example, if you want to find the odds of flipping 4 heads before a tails, you would find the odds of getting HHHHT, which is simply $.5^5$.

Take a look at the following distribution:

From this, we can see the probability behind each number of tails before getting a heads. The interpretation of $0$ is that the first flip was a heads. Again, if we were using an unfair coin, the distribution might look something like this:

Poisson Distribution

The Poisson distribution models the number of fish you can expect to catch in a given period of time. Just kidding. Kind of. This distribution was named after the French mathematician Simeon Poisson, whose last name translates to “Fish.” It models the expected number of events in a certain period of time given the average number of observed events. For example, if you know that you pass 3 gas stations every half-hour on a road trip, and you wanted to determine the probability of passing 5 in the next half-hour, you might use a Poisson distribution.

It has one parameter: $\lambda$. This paramter is the average number of successes for the specified time period. The pmf is defined as

$P(\lambda, k)=\frac{\lambda^ke^{-\lambda}}{k!}$

where $k$ is the desired number of successes per period.

We’ve officially passed into the realm of pmf’s that I don’t understand. Intuitively, you can derive this from the binomial distribution by setting up a formula which represents a number of discrete probability chunks, each with probability $\frac{\lambda}{n}$ (where $\lambda$ is the average # of events per time period and $n$ is the number of discrete chunks). A binomial distribution models the expected number of successes given a certain probability and number of trials, so we can substitute this probability into a binomial limit as $n\rightarrow \infty$. After some complicated math with limits and $e$, we get the pmf of a Poisson distribution.

Below you can see a Poisson distribution for $\lambda=1$, $\lambda=2$, and $\lambda=6$.

Many of these discrete distributions have analogues in continuous distributions (for example, the geometric distribution and the exponential distribution). I chose not to cover the continuous distributions because I was already at least conceptually familiar with them, but maybe I’ll dive into their probability functions in a later post.

Concluding Thoughts

Well, that covers a lot of the things I’ve been learning related to probability. I wanted to cover some other interesting stuff like continuous distributions and the Central Limit Theorem, but I didn’t think I would do them justice. This post was mostly the product of a day and a half, so I could only get so detailed with some of the formulae. I might revisit this topic in a future blog post or after I’ve actually taken a class on the matter. Anyway, thanks for reading. I need to go—I have an upcoming Pokémon tournament to prepare for 😈

The Gholdengo Experience

Wed, 07 May 2025 00:00:00 +0000

I recently attended the 2025 Milwaukee Regional Championship for the Pokémon trading card game. Going into this tournament, I had one goal: to day 2. Large Pokémon tournaments (regional or international) are organized such that the first day of play consists of 8 or 9 rounds of swiss bracket: players with the same win record play each other in best-of-3 matches. At the end of the first day, the top players (typically the top 10% or so) are invited to day 2, where the tournament is concluded via additional rounds of swiss and a final top cut.

In wanting to day 2, I am wanting to place in the top 10% of a group of serious players who all have traveled from large distances to be at this regional tournament—not an easy task. Still, I think it’s achievable. I had only played in one prior regional, but I was one win away from day 2-ing—and I am a much more experienced player now than I was at the previous event.

Deck choice & meta calls

The first step to doing well at a big tournament like this is selecting a deck that you believe will give you the best chance of success. For a few reasons which I will outline later, I chose Gholdengo Dragapult as the deck for the regional.

The strategy behind this deck is to run 1 of every type of basic energy card, search them out with Energy Search Pro, and swing for huge damage numbers using Gholdengo ex’s Make It Rain attack. This deck mainly attacks with Gholdengo, but also runs a thin line of Dragapult ex for draw power and as an additional attacker. Pokémon like Cornerstone Mask Ogerpon are a huge issue for Gholdengo, and having the option to attack with an additional Pokémon can make a huge difference in some matchups. Dragapult only needs 1 psychic and 1 fire to attack—both of which are already being run in the deck.

To me, this deck felt like a good meta call for the following reasons: I could finish out games quickly, and the new up-and-coming deck Joltik Box has a pretty bad matchup into it. The other deck I was considering for the event was pure Dragapult: a deck which slowly locks out your opponent and tries to make a comeback by disrupting them and swinging with the huge health attacker of Dragapult. Joltik box was a deck created in response to this strategy. It aims to power up an Iron Hands ex and start KO’ing your small Pokémon for 2 prizes each. I didn’t want to face down these hyper-turbo aggressive decks with Dragapult. In addition, Dragapult games can take a long time, and in a tournament where you only get 50 minutes to play 3 games in a match, I wanted the ability to close out games quickly. Thus, I landed on Gholdengo.

Gholdengo had a good matchup against Dragapult (one of the most popular decks) but didn’t have the best matchup against some of the decks in the format like Raging Bolt. I figured I wouldn’t see many of them. Spoiler: this was not the case…

Round 1: Tie against Charizard

Let’s walk through each round of the tournament and see how I did! The first round was against a pretty new player on Charizard—one of the worst matchups for my deck. Not a good start. This deck aims to utilize cards like Dusknoir and Briar to manipulate prizes and win faster than your opponent is able to. Now, the guy playing this deck was obviously fairly new, so I wasn’t too worried, but I still had to be careful.

The first game was an easy win: he got off to a slow start and I was able to run him off the board. The second game was not. I prized Energy Search Pro. Remember how Gholdengo needs a lot of energy to discard in order to be able to swing for big damage? Without the energy search pro, there’s no way to aggressively search those energy out of the deck. So, I was in a predicament. Should I just scoop (forfeit) the second game of the match and try and win the third one? Or should I stick it out and attempt to mount a comeback? I opted for the latter—I didn’t want to scoop against a new player like this and I figured leaving the match to the last game was a scary risk. Unfortunately, I was unable to win, and we moved on to the third game. This was a heartbreaker—time was called during our game, and I was 1 turn away from winning when the tie was officially announced. In tournaments like these, if time is called during a game, each player gets an additional two turns, and if neither player has won at the end of the turns then it is officially a tie.

I was quite frustrated with the tie: I knew that it would hurt my resistance later (the guy went 1-5 drop) and it certainly wasn’t the start I had been looking for. Still, it easily could have been a loss against Charizard, so I guess I have to count my blessings. On to the next.

Round 2: Win against Dragapult

Round 2 I faced a really good player: we both sat down and immediately talked about how disappointed we were with the first round tie. Luckily, he was on Dragapult, which was one of the decks that my list was designed to beat. In the first game, I was able to quickly take aggro and start taking KO’s before he was set up. Dragapult, as a deck, relies on a lot of hand disruption with cards like Iono and Unfair Stamp. To combat this, I run a Genesect:

This card allows me to quickly take aggressive KO’s against Dragapult without fearing an early unfair stamp, which makes a huge difference in how the matchup goes. It works like a charm, and I take quick 2-0. The second game was incredibly close, but a clutch Professor Turo’s Scenario allows me the turn I need to win the game. 1-0-1.

Round 3: Loss against Gholdengo

This round absolutely destroyed my confidence. The player I was facing was great: I actually ran into him the next day and we had a nice conversation about our matches. I won the first game in convincing fashion, but bricked (had a horrible hand) the second game. In the third game I was again off to a good start. Remember the Genesect from earlier which shuts down ACE SPECs? In the mirror match, you’re normally good to evolve and start swinging if you have a Genesect down. I did, so I evolve to Gholdengo and KO a gimmi. However, he reveals an insane tech: Klefki.

When this is in the active, basic Pokémon don’t have abilities. This means that with Klefki in the active, he is able to get around Genesect and play his Energy Search Pro. So, he takes the return KO on my Gholdengo, and the first two prizes. This means that I’m almost completely unable to win. I try to mount a comeback with Dragapult + Iono, but it doesn’t work. 1-1-1.

Klefki completely took me by surprise. I hadn’t seen this card being run in any Gholdengo list previously, so I really started to wonder if I was behind the meta for this tournament. A little worried, I headed into round 4.

Round 4: Win against Blissey

I was in the pits this game. I wasn’t feeling good about the tournament, and starting off the day at 1-1-1 was not putting me on track to day 2. So, I needed an easy win. Luckily, this is exactly what I hit.

I was facing a deck which used Blissey ex alongside cards like Munkidori to outlive and take KO’s on the opponent. Gholdengo doesn’t leave any damage on the board for Munkidori to manipulate, so theoretically I have a good matchup into this list. The one problem was Cornerstone Mask Ogerpon:

Gholdengo is unable to hit this Pokémon. So, if they leave this guy in the active and don’t play any Blissey, I am pretty screwed. However, this is where the Dragapult part of my deck comes into play: It can hit Cornerstone. However (again), I prized my Dragapult on the first game. However (again again), my opponent misplayed and benched two Blissey. This allowed me to gust and KO both for 4 prizes, and take snipe KO’s on Munkidoris with Fezandipiti ex for the last 2. Dragapult was the last prize…

Game 2 was a stomp. I didn’t prize any of the pieces for Dragapult, so he was unable to do anything with Cornerstone and I won handily. 2-1-1.

Round 5: Win against Dragapult

This round was scary for 1 main reason: The clock. Remember how I mentioned that each round is 50 minutes for a best-of-3? Well, in this round my opponent was playing incredibly slowly on Dragapult Dusknoir. I won the first game, but we only had 20 minutes for the last 2 games after this.

We start the 2nd game, and I notice that my Energy Search Pro is prized, along with 4 energy. This is a huge problem. Beating Dragapult often requires being able to piece together a KO against the first Dragapult they swing with, after they disrupt you. No Energy Search Pro means that accomplishing this is much, much harder. I was in another predicament: Do I scoop now and try to take a quick win in game 3? Or do I stick it out and try to win an unlikely comeback? I opted for the latter, which was probably a mistake. I was able to win, but only because my opponent missed a turn of attaching energy. I had to swing into a Dragapult for 150 with Gholdengo, which is never something you want to do. However, they were unable to find another energy to attach to a benched Drakloak, so after I KO’d the Dragapult the next turn, they were unable to piece together an attack. All I needed was this 1 turn of leeway, and I was able to roll the advantage to a victory. 3-1-1.

Round 6: Win against Joltik Box

I’m finally seeing the benefits from my deck choice. He opened Joltik, and I knew I was favored. My list runs a Vengeful Punch to get around Pikachu ex, and so I can normally take advantage in the prize race.

Game 1 he bricked—not much to say. However, the second game was very interesting. My opponent started with a Mew ex, and benched Miriadon to find Joltik and Pikachu ex. He then benched Fez and another Pikachu. This means that he is in a board state where he has Mew, Fez, Joltik, Miriadon, and 2 Pikachu—and no room to bench any other Pokémon. This is normally OK; you don’t want to bench more 2-prize liabilities than you have to against Gholdengo, but it’s unlikely that I will be able to gust up something 3 turns in a row, so it’s not the end of the world. However—he was bench locked. He was obviously anticipating me evolving into a Gholdengo, but while planning for this he did not set up an Iron Hands, just two Pikachu. This means that I am free to never evolve into a 2-prizer until he swings into my active Gimmighoul with a Pikachu: Something you never want to do. Thanks to this, I was able to successfully stall him for turns while he pointlessly waited for me to evolve into Gholdengo. Eventually, he reluctantly took aggro and swung into me, taking 1 prize in the process and enabling my Counter Catcher. Perfect. I was able to gust as needed to win the 2-2-2 prize race. I was really proud of this game because it was a great punish of my opponents misplay of board-locking themselves. 4-1-1.

Round 7: WIN again Poison Arch

This round was horrifying. One reason is that the deck my opponent was playing, Poison Archaludon, has the ability to win before you can even play a card. With a combo involving poison damage-modifying cards and Pecharunt, the deck can take a KO on your only Pokémon in play if they have 80 health or less: winning the game on turn 1. As my Gimmighouls have 70 health, I am a prime target for this donk strategy. Luckily, I was able to start Genesect in game 1, which is unable to be donked. I took a win when he missed a gust effect on his second turn to take a 2-prize kill.

The other reason this round was horrifying is that if I win, I officially qualify for day 2. I had clawed my win record back from the poor 1-1-1 at the start to the quite decent 4-1-1 I currently had. 5 wins and a tie makes day 2, so if I won this round I would achieve my goal.

Heading into game 2, I know that all I need to do is bring this one home. He goes first—but I start Munkidori this time. No donk potential. He goes ahead in the prize race, taking the first 2 poison KO’s before I can piece together a Gholdengo + gust response. I’m in trouble. On the turn he goes down to 2 prizes, he Iono’s the both of us to 4 cards—disrupting the large hand I had amassed via Coin Bonus, but also disrupting himself in the process. He takes the KO, and I know that I am in trouble.

I promote Gholdengo. He has an Archaludon ex in the active with a Binding Mochi. I bench Genesect, attaching a Vengeful Punch. No ACE SPECs for him. I then take a KO on his active Archaludon—but I do something smart. He is currently poisoned, and taking 30 each turn from the Perilous Jungle in play. This means that if I do 250 damage, the poison will knock him out, preventing him from using Fezandipiti’s ability on the next turn. I perform this play, leaving him with 4 cards. He needs to piece together an Arch, a Binding Mochi, and an attach for turn.

He immediately plays arch and attaches an energy for turn. All he needs is the Mochi. However: He stops to pick up my Genesect and asks “No pink cards, right?” I confirm. He sighs, and plays a Carmine discarding a Secret Box. This is huge: If I hadn’t played the Genesect on the previous turn, he would have won. He draws his 5 cards—no Binding Mochi. He promotes a 1-prizer, poisons, and passes. I now have 1 goal: Find a Boss’s Orders. I have about 15 cards in deck, a Superior Energy Retrieval in hand, and 4 draw abilities enabled. In addition, time has been called. If I don’t find this win, it’s all over. Here we go.

Recon directive 1: nothing. Recon directive 2: nothing. Coin bonus 1: nothing. At this point, I have seen 6 cards from my deck, and no Boss’s Orders. It’s down to the final Coin Bonus from the active Gholdengo. I slowly flip over the first card… nothing. I look at my opponent and flip the last card onto the table—not even bothering to hide it. Boss’s Orders. I immediately Boss the Fez and take a KO to take my last two prizes.

I have officially made day 2, and in the most cinematic way possible. Digging with Coin Bonus to find the 1 card I needed at the last possible instance: I can’t imagine anything more perfect.

Round 8: Tie against Tera Box

I’m facing Alex Schemanske this round: a professional player. He is absolutely amazing at the game, and I’m so frazzled from the adrenaline of the previous round that I immediately misplay game 1 and lose as a result. However, in game 2 I lock in: I perfectly manage the prize-state of my board and take a close game 2 after a long back-and-forth. In game 3, we’re both playing incredibly quickly. Time is almost up, and we’re both screaming through our decks. Unfortunately for him, time is called, and on his final turn he is 1 prize away from winning. I can’t take it either, and so we tie. Pretty happy to take a game off of one of the pros.

Round 9: Win against N’s Box

This round was the first of day 2. I am pretty nervous—the players who qualify for day 2 are some of the best in the world, and I don’t know how well I’m prepared to hang with them. In addition, this deck has an absolutely brutal matchup against mine: Darmanitan can hit Gholdengo for weakness, spreading massive damage across the board and one-shotting if I discard too many energy.

Game 1 goes well; he doesn’t get a quick start and I run him over. Game 2 is extremely close. We both play it very well—but he makes a major misplay on his final turn. The board state is such that if he is able to Boss and use flamebody cannon, I lose. So, I swing into a Zoroark for 250: leaving it susceptible to a Munkidori play the next turn, and turning off his Counter Catchers. I also attach a Vengeful Punch to the active Gholdengo, and he forgets about it. He swings back into me, KO’ing himself in the process, and letting me boss a 2-prizer for game. I believe he would have lost anyway to the Munki play, but his gaffe made the win easier. 6-1-2.

Round 10: Loss against Raging Bolt

No! I hit the deck I was scared of. Not only do a lot of things have to go right in order for me to win this matchup, but I only have a few turns to piece together the cards I need. I absolutely brick both games: no supporters or even ball search. I get donked the first game and run off the board the second. Unfortunate, but it happens. 6-2-2.

Round 11: Win against Tera Box

Game 1 he bricks. Game 2 was interesting: He started Cornerstone Mask Ogerpon, which is fortunate for him against Gholdengo. Hoewver, I am able to start building up Drakloaks, and after he Boss’s one of them he is unable to piece together a KO on the second, leaving me with a powered up Dragapult. By now, he is down to 3 prizes, so I Iono the both of us and swing into his Cornerstone, damaging a Pikachu ex on the bench, a fan rotom, and a hoothoot. He whiffs a Noctowl from the 3 cards he drew, and is forced to swing back into my Dragapult with the Cornerstone. I am able to gust Pikachu, move damage with Munkidori, and Phantom Dive for a 6-prize turn with Gholdengo. How often does that happen?

7-2-2.

Round 12: Loss against Raging Bolt

This was an anticlimactic ending to the tournament. Before the match, my opponent asked if I wanted to just tie to guarantee a spot in the top 100. I declined, because this was my first day 2 and I wanted to see how well I could do. This ended up being a mistake, as I megabricked the first game and got blown off the board. In the second game I actually had a chance to take the lead on the prize race with a well-timed Ciphermaniac’s Codebreaking, but my 1 Counter Catcher was prized. What a heartbreaker. I ended the tournament with a record of 7-3-2, placing 136th.

Concluding thoughts

I was very happy with my performance this tournament. I didn’t make too many major misplays: the one that stands out is not playing around Klefki in game 3 of round 3. I was fairly happy with my deck and list—I wish that I had realized how much of the meta is decks that were not Dragapult. A large portion of my 60 was dedicated to giving me great odds against Pult, but this left me weaker against hyperaggressive decks like Raging Bolt.

Still, I am proud of myself for making day 2. I think I’ll pick up a different deck for NAIC, so we’ll have to get to testing.

Grad School Scramble I: Ordinary Least Squares Estimation

Wed, 26 Feb 2025 00:00:00 +0000

I am proud to announce that I will be attending ASU in the fall to pursue a PhD in quantitative psychology under the guidance of Kevin Grimm! This decision was arrived at after a long and arduous application process wherein I flew to 7 different programs to interview. After all of my traveling, ASU was the one that felt most like home.

To prepare for grad school, I figured that it would be helpful and cathartic (and a little bit exposing) to use the blog to revisit some of the beginner topics in quantitative psychology that I am not incredibly well-versed in. Thus: Welcome to the birth of Grad School Scramble!!! In each installment of this multi-part series, I will attempt to learn and explain some area, subject, or topic of quantitative psychology that I had previously not understood well. Important disclaimer: this is a judgement-free zone! If I cover some concept that makes you say, “Hey, shouldn’t an incoming PhD student have a better understanding of this subject matter?” then, well, too bad! In this pastel-gray and baby-blue blog page, we can all find solace together in the fact that we are amateurs.

Today’s topic: Ordinary Least Squares Estimation in Linear Regression (And Other Methods)

Kevin has been doing this awesome thing where he sends me small coding assignments designed to introduce me to the field of data simulation in advance of my first semester. These assignments are typically in R, and they normally involve me simulating some data on which I perform some statistical technique. For the first one, here was the task: Generate $n=500$ where $x_{1i}$ follows a standard normal distribution (mean = 0, std = 1), and $y_i=.5⋅x_{1i}+e_i$ where $e_i$ is normally distributed with a mean of 0 and a variance of .75. It was a nice little foray into data simulation, and I also calculated an estimated slope coefficient each time the data were generated.

However, this got me thinking. To determine the coefficient I used the lm function, which states the following as one of its parameters:

weights

an optional vector of weights to be used in the fitting process. Should be NULL or a numeric vector. If non-NULL, weighted least squares is used with weights weights (that is, minimizing sum(w*e^2)); otherwise ordinary least squares is used.

Notice that? Ordinary least squares. I realized that I didn’t actually have a good handle on what the ordinary least squares estimation method was doing, so I spent some time googling it.

Background

To contextualize, let’s get a quick run-down on linear regression. (Feel free to skip this entire section if you are familiar with it.)

Linear regression, at its core, aims to estimate the level of some outcome variable, say $y$, on the level of some predictor variable, say $x$. Imagine that you have some (ambiguous) set of data like so:

You might rightly notice that there is a general relationship between variable $x$ and variable $y$: as the level of $x$ increases, so does $y$ (generally). You then might ask: is there a way, given $x$, to predict a corresponding value for $y$? This is exactly what linear regression aims to do—to create a non-deterministic formula which predicts outcomes from a given predictor. One way to do this is by fitting a line (linear regression) that follows the trend of the data:

As per our 8th grade math education, we know that this line would take the form of $y=mx+b$, where $m$ is the slope and $b$ is the intercept. If our line perfectly described the data, then we would be able to perfectly predict each level of $y$ from each observation of $x$. Unfortunately, as we can see from the fact that not all of our data points fall along the line, it is not a perfect prediction. We call this error, or the residuals. So, linear regression aims to incorporate this by adding a third variable to the equation of the line: the variance of the residuals. Let’s dissect this.

Generally, when we fit the line to the data, we expect the data points to be closer to the line than away from it. For the purposes of linear regression, we assume that they do so in a normal manner. Thus, for each level of the $x$ variable, there is a range of values we would expect for the $y$ variable.

For example, in this picture the purple curve shows the range of values we would expect from $y$ at the given point. We would expect to see the blue dot more than we would expect to see the green dot. (This assumption is called the assumption of normality of the residuals and the assumption that this range is the same at all levels is homoscedasticity.)

Regression accounts for this in the following manner: If we want to predict $Y$, we can do so by predicting it from two measures—a function of $X$, and an error term $ε$. If our function of $X$ is linear, we get the following form:

$Y_i=β_0+β_1X_i+ε_i$

where $β_1$ is basically the $m$ and $β_0$ is basically the $b$ in $y=mx+b$. The “spread” of data around the line is accounted for by $ε$.

We tracking so far? Basically, once we have values for $β_0$ and $β_1$ and understand our error, we can regress $y$ on $x$ to understand their relation.

Ordinary Least Squares Estimation Theory

The question then becomes: how do we estimate values for $β_0$ and $β_1$? Think back to our previous (data-ambiguous) example. If you wanted to draw a “line of best fit” through the data, you might come up with the line we drew previously:

However, maybe your colleague disagrees, and says that a different line (orange) actually fits the data better:

We need to come up with some objective way to state that a line fits the data best. This is where Ordinary Least Squares Estimation comes in. One way to objectively state that a line fits “best” is to attempt to minimize the distance between each data point and the line of prediction. Recall that the distance is defined as the error:

If our line fits reasonably well, we would expect the distance between the points and the line to be small. For a counterexample, a terrible line of prediction:

See how big the distance (error) is? This is the basis of Ordinary Least Squares: the smaller the overall error, the better.

Soooo… this is easy, right? Just minimize the total amount of error! Well, there’s one problem. If we have a large positive distance between a point and the line, and we have a large negative distance between another point and the line, they cancel out, and the overall error is 0. So, we instead square the distances so that we can meaningfully compare them. If we can find values for $β_0$ and $β_1$ that minimize the squared error, we have our line.

Ordinary Least Squares Estimation Calculation

All the previous description of linear regression was assuming that we knew population values for our data. In reality, we always work with samples. So, we transform our previous population equation $Y_i=β_0+β_1X_i+ε_i$ to a sample-specific equation:

$y=b_0+b_1x+e$.

Well, we might notice something. In this equation, $y$ is the actual data point that we observe. But we also have access to another version of $y$: $ŷ$. This symbol represents the predicted value of $y$ for a given data point—the value for $y$ we would expect without error. Thus, we also have the equation $ŷ=b_0+b_1x$ by definition.

Yay! Now we can do some algebra.

$y=b_0+b_1x+e$ and $ŷ=b_0+b_1x$. Thus,

$y=ŷ+e$

$e=y-ŷ$

Well now! Recall that we’re trying to minimize the residuals for our line of prediction. Now we have a definition for them. According to our previous theory, we need to square this term to meaningfully compare observations:

$e^2=(y-ŷ)^2$

And all we need to do is sum these babies up and find the values for $b_0$ and $b_1$ that minimize that sum!

$∑(y-ŷ)^2$

Example: Ordinary Least Squares Estimation with Three Data Points

Let’s illustrate this calculation with an example (which I came up with myself, so bear with me). Imagine that we have 3 data points, and we want to estimate the slope and intercept of a prediction line via our previously-discussed method. We’ll say these points are $(1,2)$, $(2,3)$, and $(4,6)$.

Recall our previous formulae: $y=b_0+b_1x+e$ and $ŷ=b_0+b_1x$. We want to solve for the values of $b_0$ and $b_1$ by minimizing the sum $∑(y-ŷ)^2$. So, let’s use some calculus.

How can we find the local minimum of a function? By calculating its derivative and determining when the derivative is equal to $0$. Remember, the derivative of a function at a certain point represents the rate of change of the function at that point: in other words, the slope of the tangent line at that point. So, if we know that the slope of the tangent line is $0$ (and that we’re looking for a minimum), we know that the function is going from decreasing to increasing, and we have found a local minimum.

So, let’s calculate the sum $∑(y-ŷ)^2$ and find its derivative!

$∑(y-ŷ)^2$ will have 3 different components: one for each point. Let’s label these $(y_1-ŷ_1)^2$, $(y_2-ŷ_2)^2$, and $(y_3-ŷ_3)^2$ where $point 1 = (1,2)$, $point 2 = (2,3)$, and $point 1 = (4,6)$.

Remember, $ŷ=b_0+b_1x$. So for each observation we can plug the actual point’s value of $y$ in for $y_i$ and plug in our variables of $b_0$ and $b_1$ for $ŷ_i$. This will let us get a function in terms of $b_0$ and $b_1$—the values we want to solve for.

$(y_1-ŷ_1)^2=(2-(b_1+b_0))^2$

$(y_2-ŷ_2)^2=(3-(2b_1+b_0))^2$

$(y_3-ŷ_3)^2=(6-(4b_1+b_0))^2$

In each equation, we put the x-value of the point alongside the variable $b_1$ because our predicted value $ŷ$ is a function of $b_0+b_1x$.

Now, we mash all of these components together:

$∑(y-ŷ)^2=(2-(b_1+b_0))^2+(3-(2b_1+b_0))^2+(6-(4b_1+b_0))^2$

Next is finding the derivative. Let’s expand this equation so we’re not dealing with exponents around parentheses (I’m gonna gloss over a few intermediate steps).

$∑(y-ŷ)^2=4-4b_1-4b_0+b_1^2+2b_1b_0+b_0^2+9-12b_1-6b_0+4b_1^2+4b_1b_0+b_0^2+36-48b_1-12b_0+16b_1^2+8b_1b_0+b_0$

$∑(y-ŷ)^2=21b_1^2-64b_1+3b_0^2-22b_0+14b_1b_0+49$

Awesome! Now we have a more manageable function with two variables. We want to find the minimum of this function with respect to both, so we take the partial derivative with respect to $b_1$ and $b_0$. (Again, skipping some intermediate steps.)

$\frac{d}{db_1}=42b_1-64+14b_0$

$\frac{d}{db_0}=6b_0-22+14b_1$

Ok. Home stretch. Now we set these two equations equal to 0:

$42b_1-64+14b_0=0$, and $6b_0-22+14b_1=0$. We can set up a system of equations:

$42b_1+14b_0-64=0$

$14b_1+6b_0-22=0$

Which we can actually transform into a matrix by putting the constants on the right-side of the equation:

\[\begin{array}{cc|c} 42 & 14 & 64 \\ 14 & 6 & 22 \end{array}\]

$\overset{3r_2}{\longrightarrow}$

\[\begin{array}{cc|c} 42 & 14 & 64 \\ 42 & 18 & 66 \end{array}\]

$\overset{r_2-r_1}{\longrightarrow}$

\[\begin{array}{cc|c} 42 & 14 & 64 \\ 0 & 4 & 2 \end{array}\]

So, the solution to the system of equations which sets both derivatives equal to 0 includes $b_0=\frac{1}{2}$. We can plug this value into the first equation to obtain:

$42b_1+14(\frac{1}{2})-64=0$

$42b_1+7-64=0$

$42b_1-57=0$

$42b_1=57$

$b_1=\frac{57}{42}$

$b_1=\frac{19}{14}$

You get to see all the steps that time. Well, OK! Now we know the values for which our function $∑(y-ŷ)^2$ has a local minimum! So, if our calculations are correct, we should be able to plug those values back into the formula $y=b_0+b_1x+e$ to obtain a slope and intercept coefficient for our regression.

Let’s try it:

$y=\frac{1}{2}+\frac{19}{14}x$:

Hey that looks pretty good! Like, really good! I made the starting numbers up on the fly, so I’m happy to see that we got an equation with nice simple fractions and not complex decimals.

The final part of calculating the linear regression is to get the variance of the residuals (I’m gonna speed through this). Using our formula for variance we can see that $\hat{σ}^2=\frac{1}{n-p}∑\hat{ε_i}^2$ where we divide by $n-p$ as the number of degrees of freedom for $p$ parameters. We can recall that $\hat{ε_i}=y_i-ŷ_i$ by definition, and use our new slope and intercept to calculate $ŷ$ for each observation.

For $(1,2)$:

$y-ŷ=2-ŷ=2-(\frac{1}{2}+\frac{19}{14}(1))\approx0.143$

For $(2,3)$:

$y-ŷ=3-ŷ=3-(\frac{1}{2}+\frac{19}{14}(2))\approx-0.214$

For $(4,6)$:

$y-ŷ=6-ŷ=6-(\frac{1}{2}+\frac{19}{14}(4))\approx0.071$

We can sum the squares of these $3$ values: $(.143)^{2}+(-.214)^{2}+(.071)^{2}\approx0.071$.

Finally, we divide this value by $n-p=3-2=1$. Dividing by $1$ does not change the number, and so our $\hat{σ}^2=0.071$.

So, all together, we have the following formula for our linear model: $y=\frac{1}{2}+\frac{19}{14}x+e$ with $\hat{σ}^2=0.071$.

Verification

Let’s see how we did. Here is code which runs it in R:

x <- c(1, 2, 4)

y <- c(2, 3, 6)

d <- lm(y~x)

summary(d)

(summary(d)$sigma)**2

And here is the output:

Holy mackerel. That looks great. Notice that under “Coefficients” we can see the estimate for the intercept at $0.5000$, which is exactly what we predicted. Similarly, the estimate for the slope is $1.3571$, which is approx. $\frac{19}{14}$. Spot on.

Also, notice the residuals: $0.14$, $-0.21$, $0.07$. Do those numbers look familiar? Finally, we can observe the sigma**2 value at the end, and notice that it is incredibly close to the value we predicted of $0.071$.

WE DID IT!!!!!!!!

Conclusion

So there you have it. How to hand calculate ordinary least squares estimation in a linear regression. Much like all my blog posts, this ran on far longer than I originally meant it to. Congrats if you stuck around till the end. For the next installment of Grad School Scramble, I’ll try to keep it more theoretical and less concrete. I don’t like wrangling $\LaTeX$.

College, in Retrospect

Mon, 30 Dec 2024 00:00:00 +0000

Boomer! I recently graduated from the University of Oklahoma with my B.S. in Psychology with Honors, Summa Cum Laude, with Minors in Computer Science and Music, and a certificate in Applied Statistics. Apparently, the longer it takes you to say your degree and all relevant qualifications, the more impressive it is, so that’s why I get to look really pretentious whenever I list it on an application. (No fair! Multidisciplinary Studies: Concentration in Public Promotion & Social Perception majors have such an advantage!)

As I recover from my typical post-semester mental collapse (read: playing the Pokémon video games for 11hrs/day all week), I thought it would be good to reflect on the different stages of my college journey and consider the events that led me to the place I’m at. Buckle up.

The beginnings

What drew me to OU in the first place? Basically, money. The first big thing to realize about me is that I was not always the shining exemplary student that I obviously am today. In high school, I didn’t place much weight on the college admissions process. I didn’t totally understand the gravitas of accepting an offer, and I didn’t have a comprehensive grasp of the full cost of tuition, loans, and living fees. So, I took it easy. I tossed my hat in the ring with big name schools like Stanford and MIT (wait-listed, technically), but didn’t try extremely hard to get in. I didn’t even tour anywhere - other than briefly visiting OU for a piano audition. I wanted to both be a therapist and continue playing, so my listed degrees were psychology and piano.

I got an offer from OU, but I kept dragging my feet about accepting. The longer I put it off, the sweeter the deal got. They kept eliminating various costs like the application fee or first-semester fees, so I just kept waiting to see how much money I could save by procrastinating. Finally, the day that responses were due, I formally accepted the offer from OU due to the National Merit scholarship package and acceptance to their piano program.

The death of the piano dream

To be fair, I technically lasted more than 1 semester as a piano major. After my first juries, my teacher Dr. John Patrick Murphy sat me down and asked if I fully understood the standards for daily practice expected of piano majors throughout college (Spoiler: I did not). By your junior/senior year, around 4 hours/day piano practice was expected. Some of my friends (read: Aidan Sudler) may have been talented enough to exist in such an environment, but I who was struggling to even practice 1hr/day was not. So, I decided to relegate the study of music to a vocal minor, and focus on psychology.

The first year of college also held many non-academic developments that were incredibly impactful. Notably, I discovered a wealth of social connections and amazing friends that uplifted and enriched my freshman year. I started college in 2020: right in the peak of COVID. During this time, most classes were online, and many in-person events were shut down. Luckily, I had groups such as:

The guys in my dorm hall (Couch 6W)
The girls on the other side of the dorm floor
The David L. Boren dorm people
The guys in Sigma Alpha Mu (Go SAMMY)
The other music students
The people in Campus Activities Council
The people who worked in the Gender + Equality Center

All of these groups were incredibly influential in shaping my first-year college development. Socially, many of the first groups listed helped me feel welcomed and accepted on campus. Professionally, my interaction with the last two groups paved the way for involvement that would occur my sophomore/junior year.

Many of the first things I did on campus were either for or concerned with the frat Sigma Alpha Mu. As a freshman, I was adamant about not joining a fraternity. I did not have a positive perception of frat guys, and didn’t jive with the flavor of masculinity expressed in many greek spaces. However, I probably watched Monsters University too many times, and eventually became curious about greek life. What eventually convinced me to join was the fact that a new fraternity, Sigma Alpha Mu (or Sammy, for short) was getting founded on campus the year that I was rushing. The representative from SAM headquarters who was founding the chapter at OU marketed it as a frat for non-frat-guys, where the founding fathers would get to play a role in shaping a culture in the fraternity that was different from many other houses. I figured this sounded cool, so I joined as a founding father.

(Slight post interruption! I’m actually writing this next part from a coffee shop in downtown Budapest while sipping the most delicious espresso. Everything prior was written in the London Heathrow airport during a layover. Ok, travel update done - back to my schooling)

One of my favorite things Sammy did together was OU Scandals. Scandals is the name for OU’s yearly student-produced musical production (no, the name Scandals didn’t result from all of the drama that theatre kids love to make). Each year, various groups (typically fraternities and sororities) self-produce a short, 15-minute musical which is then performed at a large showcase. The cast of each show consists of a pairing of two specific groups (e.g., the sorority Gamma Phi Beta and the fraternity Pi Kappa Alpha), and the directors and leads are individual members from each group. When Sammy did Scandals, we paired with OU Class Council - a student government organization that performs some function I still don’t totally understand. I was both one of the artistic directors and one of the leads for our show, so I was tasked with writing some of the music and performing the role of the villain (my favorite song was an evil rendition of “You’re Welcome” from Moana). Overall, it was an incredibly fun experience, and I met some great people during the production.

That year, I also joined the leadership team of OUr Mental Health, a club dedicated to raising awareness and providing resources for mental health on campus. One of my personal highlights was facilitating a stress-relieving meditation night for a group of students during dead week. One girl fell asleep, so I consider my meditation facilitation a success.

2020-2021 Notable Classes/Professors:

Piano: Fresh/Soph with Dr. John Patrick Murphy

Elements of Psychology with Dr. Jenel Cavazos

Aural Skills with Dr. Sarah Ellis

Gateway: Fine Arts with Dr. Johnnie-Margaret McConnell

Music in Culture with Dr. Zoe Sherinian

The campus life explosion

I still don’t know exactly what got into me, but the next two years of my college career were marked by my full immersion into any form of campus involvement I could find. There was a sort of snowball effect: the more things I got involved in, the more opportunities I would hear about. One of the things I love about OU is that many of the programs and events on campus are fully student-led, so there are nearly endless ways to get plugged in.

One of the major organizations I got involved with this year was the Gender + Equality Center (GEC). Holy moly, these people were amazing. All incoming freshmen at OU are required to take a federally-mandated consent training, and I had a great experience with mine. I was a bit of a snot-nosed incoming student, and I went into the training expecting to poke fun at it and not take it too seriously. However, the peer-educator moderator (Abby Tow, who later became one of my good friends in the GEC) was phenomenal, and I learned a lot about consent and how it is communicated. So, the next year I interviewed to become a peer educator for the GEC and was eventually accepted.

If I had to pick one organization that helped me develop professionally during my time at OU, it would be the GEC. All of the people I worked with and all of the bosses I had were amazing and really cared about their employees (Bliss Brown, Jerry Lessley, Quan Phan, Crissy Brown, Amber May, Erin Simpson, Ann Schaefer, to name a few). The team of peer educators was full of motivated students and I formed some lasting friendships. Overall, I can’t recommend it enough.

Alongside the GEC, I also got involved with Campus Activities Council (CAC). Remember earlier how I mentioned that most of the organizations at OU are student-led? CAC is the main hub for many of the major events, like OU Film Series and OU Concert Series (OU Scandals is also based out of the CAC). During the fall semester, I was an exec member of CAC College Bowl, an organization that hosted a large trivia tournament each year. I got involved with College Bowl because many of the girls I was friends with in the dorms were members, and it was a blast. Much like the GEC, the team of College Bowl exec members was super fun and close-knit, and the process of setting up and hosting the tournament was full of fun ideas and events.

In the spring, I was a vice chair for CAC Fall Family Weekend (FFW): the organization that hosts the yearly family visitation weekend at OU. I was in charge of reaching out to various other organizations to host events during the weekend, but the fantastic FFW chair Sarah Stringfield honestly did most of the work for me. It was fun, but I definitely gained a reputation for slacking off. 8/10.

Finally, I also was a Teaching Assistant to Dr. Jenel Cavazos for Elements of Psychology 1113 and to Stephanie Oakes for First-Year Foundations of Psychology 1002. Being a TA was great fun, and helped me gain experience in the classroom that was not purely being a student. Highlight of TA’ing: when I planned to host a end-of-the-year party for my group in Elements of Psychology, and was told that if I actually hosted an event with alcohol for undergraduates I would get fired and barred from TA’ing again. #TeamMilgramForLyfe

2021-2022 Notable Classes/Professors:

Men’s Glee Club with Dr. David Howard

Research Methods with Dr. Scott Gronlund

Men and Masculinity with Dr. Rodney Bates

Introduction to Personality with Dr. Eugenia Fuenzalida

Beginning Spanish with Christian Doug

Research rumblings

Uh oh… you hear that? It’s the sound of the typical upperclassman realization that graduation is now sooner than later, and that you should probably figure out a semblance of a plan. Junior year is when I started reevaluating whether or not I actually wanted to be a therapist. It was sort of my “mid-life crisis” of college; I took cognitive psych and wanted to be a cognitive psychologist, I took virtue ethics and wanted to start a philosophy degree, I started working for my first research lab and REALLY wanted to do research. At the end of the day, this is the year that I first took Psych Test & Measurements with Dr. Robert Terry, and you only need read my graduate school application essays to know how pivotal this class was in getting me excited about quant psych.

My first research experience was working with Dr. Scott Gronlund in the OU Memory Lab. It was very fun! I was in charge of running participants for an experiment that aimed to determine how object memories were stored in the brain. Participants were given a list of items (objects like chairs, tables, lamps, etc.) to memorize, and they were asked to recall if certain inages (other objects) had been present in the list. The catch is that the cue images were either exact replicas of the object or the same TYPE of object (e.g., a lamp) with subtle variations. The theory is that if there was no significant difference in levels of observed change then people were storing objects conceptually rather than as a flashbulb image memory. Cool stuff.

My second research experience, during the spring semester, was working under Dr. Edward Cokely in the OU Decision Psychology Lab. The content was very cool, but I didn’t once interact with Dr. Cokely himself. He was completely hands-off with the undergraduates in the lab and tasked his grad students with managing us. I thought that was a little bit over-the-top, and I was sad that I never got to ask him questions about the material. The framework for this lab was that there exists a measure of intelligence–numeracy–that is more predictive of good decision making than any other measure of intelligence. This lab was an interesting experience. At the time, I didn’t have the vocabulary to express my questions, but I wresled with the concept of divergent validity with regard to numeracy and other measured forms of intelligence. I was also constantly curious about the methodology for how they determined that numeracy training makes someone a better decision maker overall. In the end, the experience turned sour as I accidentally missed showing up in the lab for a participant during dead week and was asked to step down as an assistant. Shhh! Don’t tell the people that read my CV!

This was the point at which I discovered quantitative psychology. During the spring of this year, I took Psych Test & Measurements with Dr. Robert Terry, which introduced me to the field. I spent a lot of time after class asking him questions I had from the other research I was involved in, and he eventually invited me to participate in OU’s Academic Success Center Research Team. I’ll get into specific quant stuff later, but this was my first foray into psych data science.

In terms of campus involvement, I was still plugged in. One of my favorite organizations I was a part of was Camp Crimson, for which I served as a Camp Guide. Camp Crimson is OU’s largely student-led freshman orientation camp, responsible for introducing new students to campus and hosting events before the semester such as Class Kick-off. Camp Guides are the individuals who run around making sure everything is running smoothly. We also get to do fun stuff such as being the hype men for competitions and leading excursions to attractions in OKC.

I continued my involvement in CAC and the GEC. At this point, I also started trying to make professional use of my involvement, so I volunteered to be a facilitator for peer support groups that met on campus to discuss stress and other facets of being a college student. It was very fulfilling. The ironic part about these groups, however, is that I honestly got as much use out of them as the students among whom I was supposed to be facilitating discussion. At this point in my college career, the pressure of everything I was involved in really started to affect me stress-wise, and I made the conscious decision to dial down the involvement for my senior year.

2022-2023 Notable Classes/Professors:

Biblical Literature with Dr. Jill Hicks-Keeton

Sociology of Family with Matthew Bejar

Virtue Ethics with Dr. Adam Green

Psych Test & Measurements with Dr. Robert Terry

Voice with Matthew Corcoran

Honing in

Senior year is where I really started to focus on pursuing the aspirations that had developed in the last few semesters. The spark having been lit by Dr. Terry, the field of quantitative psychology was calling to me like no subject had ever before. As I conducted more research with my professors, I realized how much I might enjoy doing it as a career. Being able to watch them operate in the academic environment slowly became something that embodied where I wanted to be someday.

One funny story: As I started my senior year, I decided to try and attain a computer science minor during my last year (and a half) of college. I had been missing math and coding (shockingly) and I wanted to fold it back into my education. The funny thing about this ambition is that I was not even close to having the required math credits to start the CS track. CS starts with Calculus as a co-requisite for many coding classes, and the pre-requisites for Calc 1 are trig, geometry, alg, etc. The problem was: the psychology degree at OU only requires VERY basic math classes, and the only one I had taken was a “functions for the social sciences” class my first year that probably didn’t even count for credit in any program other than psychology. I had only gotten a 2 on the AP Calc BC exam in high school, which didn’t count for any credits either. So, I was stuck. I started emailing around, and the only way I was going to be able to take calc that semester was if I took the incoming student ALEKS math placement exam, which is the test given to freshmen who want to test out of intro-level math classes. The culmination of all of this context is that I ended up having to cram study the ENTIRE math sequence–alg, geometry, trig, and pre-calc–in a matter of 3 days to prepare for a placement exam. I am proud to announce that I successfully ended up relearning the entire high school math curriculum in 72 hours, and scored into the highest category of placement. So, I was able to start work on the CS minor.

I did Camp Crimson during the summer (OK WHO? OKU). Other than that, I wasn’t active in any organization on campus other than the GEC. This allowed me to focus heavily on research.

The bulk of my work focused on my honors thesis. At OU, in order to graduate with honors you must complete a thesis during your senior year over a topic of your choosing. I wanted my thesis to be in the form of a published research paper, so I reached out to and worked with Dr. Jenel Cavazos to plan, organize, and complete a full paper throughout the course of the year. The paper focused on AI use among college students; more information about it can be found here. More than anything, this project allowed me to realize that I wanted to continue my education into a PhD. The process of conducting review, developing a study, assessing results, and authoring a paper made me feel like I was at home, academically. I loved it–and learning that exact experience constituted a majority of grad school was enough for me to fully commit to the idea.

Alongside my senior thesis, I also conducted work on student retention data with Dr. Terry. I was tasked with assessing methods of handling missing data such as FIML and Multiple Imputations with Chained Equations. I recognize that the length of this post is starting to approach the upper limit of what I was hoping for, so I’ll forego explaining all the research specifics. Information about any particular project can be found here.

Overall, this year really was just a bunch of academics. It was SO, SO fun, though. More than anything, it opened my eyes to the opportunities that are available to individuals pursuing research. Some of my favorite memories from the year are from presenting my paper at conferences like SWPA or OPS (I won ‘Best Undergraduate Paper’ at one!). It helped me solidify my certainty that I was on a good path for me.

2023-2024 Notable Classes/Professors:

Great Books of Western Civ with Dr. Robert Lifset

Psychology Capstone with Dr. Robert Terry

Discrete Structures with Dr. Lauren Grimley

Stat Models of Tests Scores with Dr. Robert Terry

Computer Organization with Dr. Mohammed Atiquzzaman

The home stretch

After all of the previous exposition, my super-senior semester didn’t really bring any new revelations. It was much a continuation of the previous developments, both academically and personally. My victory lap.

I realized this semester that I technically qualified for a certificate in Applied Statistics (which is basically a minor? Still not sure exactly what it means) due to all of the math/stats courses I had taken. One more qualification on my degree! Yay!

Alongside continuing some of the research I had previously done such as my work with Dr. Cavazos and work with Dr. Terry, I got involved with some oddball projects you might not expect. I started working with Dr. Lauren Grimley, one of my math professors, on math research involving the game of peg solitaire. The way I got involved on this project was as follows: I had Dr. Grimley for the class Discrete Structures the previous year, and I thought the class seemed really cool. I needed an honors credit still to graduate with honors, so I asked her if I could do an extra paper/project for the class to qualify it for honors credit. She seemed receptive to the idea, so we met to discuss potential ideas. After a while, she introduced me to this side project of hers looking at the solvability of various classic board games: peg solitaire being one of them. Through the project, we analyzed peg solitaire in 3 colors: the specifics of which can be found on my research tab. This was another project that resulted in a paper, which I am incredibly proud of. (One funny story: the Discrete class that Dr. Grimley taught was so difficult that the students staged a full revolution. The dean was so overwhelmed with complaints and with students visiting his office that he had to announce that something would be done and that they would investigate the class. For the record, I am completely on Dr. Grimley’s side, and think that the students were just complaining too much.)

Finally, I also worked for Dr. Eric Day to help modify a video game that he used for a lab assessing exploration and exploitation patterns in workplace learning. Again, I don’t want to just blandly describe all the various projects I was working on, but if you want to try a version of the game for yourself you can visit the following link: https://deeper-paths.firebaseapp.com/.

I could not have been happier with the classes I took my final semester. I took Linear Algebra (incredibly engaging), and also TA’d for Advanced Undergraduate Statistics taught by Catherine Bain (one of the graduate students who helped with my honors thesis, among other things). Alongside those classes, I also took Chinese Religions with Dr. Geoffrey Goble. Oh boy. What an adventure. This class was structured such that the second half of the semester was purely dedicated to a massive roleplaying game that took place in-class. The main focus of the class was assessing the impact of the religions of Confucianism, Daoism, and Buddhism throughout Chinese history, and the game featured factions of representatives reenacting the Huichang Persecution of Buddhism court debates circa 841. I was the leader of the Confucian faction - wise, arrogant, and traditional. We argued for the restriction and abolishment of Buddhism throughout the Tang Dynasty, as such foreign ideas were obviously dangerous to the peace and stability of Our Kingdom. Not to brag too much, but our faction won the game, and I achieved the title of “winning character.” Long live Niu Sengru.

Overall, it was a great final semester to close the book on my college journey.

Fall 2024 Notable Classes/Professors:

Linear Algebra with Dr. Elizabeth Kelley

Advanced Undergraduate Statistics with Catherine Bain

Chinese Religions with Dr. Geoffrey Goble

Now what?

Well, dang. That’s my college experience.

I’m not certain what the future will hold. I’m currently in the final chapter of the grad school application process: Next week I start flying out to schools that have invited me to visit. I currently have next-to-no idea on the heuristic I should be using to select a program, so I’m hoping these trips will bring some much-needed clarity. (Plus, the schools all pay for my flights/lodging. Nice!)

Would I have done anything differently throughout college, in retrospect? Probably. I would have loved to study abroad for a semester—the price tag always kept me away. I think it would have been worth it, though. Other than that, I’m honestly very happy with how it went. OU was a great school—I met some great people. I grew a lot, and I found a passion in quantitative psychology. Not sure you can ask for more than that.

Time to see what grad school is like!

An Intuitive Method of Interpreting Quantile-Quantile Plots

Wed, 11 Sep 2024 00:00:00 +0000

This semester, I have the honor of being a teaching assistant for Advanced Undergraduate Statistics. Despite TA’ing for the course, I have never actually taken it myself. I had worked with the instructor (the fantastic Catherine Bain) on prior research, and she figured that I was already knowledgeable enough in the material to help teach it (Oh, if only she knew how inept I actually am).

One of the consequences of her gracious assumption is that there is a lot of material in class that I have attemped to learn through guerrilla tactics, but have not had formal training in. As such, we often cover material in class that fills in some basic gap of knowledge that I hadn’t realized existed.

Other times, we cover material in a more structured manner than I had previously experienced, which is always helpful. Overall, it’s been a great way to more fully immerse myself in basic statistical methodology.

During our last class period, we completed a lab where the students had to work with a dataset and clean it of missing values and outliers. As is common, testing for outliers involved checking the data for normality. For the purposes of undergraduate statistics, this consisted of generating a histogram of the data and visually assessing it.

The data were the individual means of a 6-item 5-point scale measuring general parenting behaviors from 1 = coercive parenting to 5 = positive parenting. As we can see, the data seem to have a negative skew. While we discussed it in class, I wanted to introduce some of the students to a more formal way of assessing normality: Quantile-Quantile plots.

Keane’s quick-and-quality Quantile-Quantile plot guide

A Quantile-Quantile plot (Q-Q plot, for short) is a relatively quick way to assess the normality of a distribution. A Q-Q plot is a scatterplot which plots two sets of quantiles against each other: theoretical vs. sample.

What is a quantile? Well, a quantile is basically another way to say “percentile.” When you plot quantiles, you are plotting the points at which a certain amount of data fall below (50%, 90%, etc).

A z-score normalizes data by expressing its distance from the mean in terms of standard deviations. When we plot quantiles, we plot them in terms of their z-score. As such, the x-axis of a Q-Q plot often ranges from around -3 to +3, which gives us a range of data of about 3 standard deviations from the mean. The y-axis of a Q-Q plot is often the numerical value of the data.

When we look at a Q-Q plot (shown below), it shows us the expected distribution of quantiles (if our data were normal) plotted against our actual sample of data. If our data were perfectly normal, they would follow the expected distribution perfectly linearly. The blue line on the plot below displays this expected relationship.

From this, we can evaluate any deviation from the expected line to mean that our data deviates from normality. If this is confusing, don’t worry — I’m writing this because it confused me too.

As we can tell from our quick-and-easy Q-Q plot guide, Q-Q plots are useful in assessing the normality of data. Prior to yesterday, this was about the extent of my knowledge of them. However, as I was thinking about them in class, I realized that I didn’t fully understand why we interpreted them the way we do.

For example, by running a Q-Q plot on the data previously observed (in the histogram), we obtain the following plot:

As we can see, our data deviate from the expected distribution. Notably, our scatterplot shows a curve, which (apparently) means that our data are skewed. But, why?

I realized that I had no real knowledge of the theoretical basis for Q-Q plot interpretation. Part of this problem originates from the fact that online materials for learning about Q-Q plots suck. Most online sources do a great job of explaining what a Q-Q plot is, but fail to delve into proper interpretation. For example, this website explains very well what they are, but simply lists the different possibilities for Q-Q plot outcomes. Even UVA (a school I’m applying to) on their website don’t explain why we see differences between curved plots and extremity-deviated plots.

So, I asked Catherine, who ended up being a great help. One key part of intrepreting a Q-Q plot, I realized, is thinking about it in terms of individual data points. Take a closer look at the previous graph:

Let’s think about what the data point circled in red is communicating. On the x-axis, we have our theorized z-scores for perfectly normal data. On the y-axis, we can see that the point’s value is around ~1.2. So, what the heck does this mean? Well, we observe that the point is distant from the line. Now consider what the line is telling us: the expected relationship between our data and the normal distribution. So, we can imagine our line extending all the way past the left side of the graph, like so:

From this, we can see that if our line continued, we would expect to see the value of 1.2 at approximately a z-score of -5. Since our score of 1.2 occurs around a -3 z-score, it appears at a frequency about two standard deviations higher than expected. Thus, it (and all the scores under the curve around it) is observed more than we would expect from a typical normal distribution.

How do we synthesize this with our understanding of normality? Well, this highlights another problem with how normal distributions and skew are communicated. Most diagrams of skewed data vs. normal data that I’ve seen look something like this:

This isn’t wrong per se, but it’s a little misleading. These distributions would no longer be matched on their respective means. For example, consider our previous example. If we take a data point at the extreme end of our negatively-skewed data (highlighted on the plot), we know that it is occurring at a frequency higher than we would expect from a typical normal curve. But wait, this appears inconsistent with the chart of skewed data we just saw! Doesn’t it look like that data point is actually below where it should be on the normal distribution? Doesn’t that mean we are seeing it less than we should?

Thus the problem. I propose conceptualizing skewed data in one of two different ways.

The first way: drawing distributions accurately

I made a slight modification on the previous graph which I think makes all the difference:

Now, if we think about a data point on the extreme end of the negative skew, we can see that it truly is falling above where we would expect it: something that our initial Q-Q plot was telling us. Most comparisons of skewed vs. normal data don’t show it to this detail, but it’s important to understand that the logistic slope of the left side of a negatively-skewed distribution (and right side of a positively-skewed distribution) decreases at a slower rate than that of a normal curve. To consider the ramifications for the right side of the graph, see the following example.

The second way: thinking about where the deviation from normality originates

Consider the following graph:

In this graph, we can see a normal distribution compared to a negatively-skewed distribution. On the tail ends of the distributions (the parts we’re concerned with in many of our Q-Q plots), we can see exactly how the data differ from normality. On the left, we see the green (negative skew) sitting higher than the base normal curve. On the right, we can see how it falls below the normal curve: it is occurring less than we would expect.

Final takeaways

That’s mostly it - I hope it’s been helpful. One additional point that needs to be made is that it’s important to remember that what it “means” for data to fall above/below the expected line in a Q-Q plot switches depending on whether the data is to the left or the right of the z-score of 0. When we considered our previous extreme data point, it was below the line on the left, meaning that it had a higher z-score than expected. However, if we consider an extreme data point on the positive end of the z-distribution, the opposite holds true: if a point is below the line, it means that it is more extreme than we would expect. This distinction comes from the fact that a z-score being “lower” means that it’s more extreme when it’s negative, but less extreme when it’s positive. Keep that in mind!

The aforementioned point influences how we can interpret data that is leptokurtic or platykurtic. If our Q-Q plot looks like a squiggly s-shaped curve, it can mean either. We just have to think about the position of our data with regard to the expected line to see which quality of kurtosis the data have.

Also, I do not claim to be an expert on Q-Q plots. I still have much to learn in my academic career. Still, I figured that I’d expound on some of the thoughts I’ve had concerning Q-Q plots these past few days.

Time to go write some grad school application essays.