January-June TTRPG Crowdfunding Retrospective Pt 2: ~D&D~

As promised, here is part 2 of the half-year crowdfunding retrospective. Check out part 1 and the raw data if you haven't already.

Warning: actual statistics contained in this article. Proceed cautiously.

D&D vs Non-D&D Projects (Averages)

Something I've long been interested in since before starting this project is how D&D crowdfunding projects fare compared to non-D&D projects. I'm well aware of high-profile high-earning projects based on 5e and on other systems (original ones, PBTA, Year Zero, Forged in the Dark, etc), but what do the data say?

Average USD raised by all campaigns, D&D projects, non-D&D projects, and AI projects

• January (75 projects)
• All: $72,133.94
• D&D: $11,479.53
• Non-D&D: $117,272.10
• AI: $5,961.51
• February (232 projects)
• All: $14,551.75
• D&D: $10,399.54
• Non-D&D: $15,904.18
• AI: $6,770.67
• March (267 projects)
• All: $21,973.59
• D&D: $27,805.14
• Non-D&D: $19,941.38
• AI: $4,479.97
• April (145 projects)
• All: $45,029.82
• D&D: $34,497.92
• Non-D&D: $54,591.68
• AI: $9,396.67
• May (151 projects)
• All: $52,807.94
• D&D: $84,426.41
• Non-D&D: $33,088.90
• AI: $7,394.13
• June (160 projects)
• All: $32,496.37
• D&D: $27,751.97
• Non-D&D: $36,003.11
• AI: $6,348.63
• Total (1030 projects)
• All: $33,355.09
• D&D: $34,115.85
• Non-D&D: $32,958.42
• AI: $6,771.98

In all honesty, this figure does not tell us all that much. Without multiple years of data, we can't be sure whether the variability seen from month-to-month is expected or not. The most important information I can share is the statistical analysis of D&D 5e vs non-D&D projects and AI vs non-AI projects.

The tests used here are two-sample t-tests assuming unequal variance. To break this down:

• t-test: statistical test used to determine whether the response of a sample or samples is statistically significant
• two-sample: this t-test compares the average of two populations to see if they are statistically different from each other
• unequal variance: the two samples are not the same size and are not assumed to have the same variation

The null hypothesis (expectation that I tested) was that there is no difference between the average money raised by projects using D&D 5e and projects that don't. As we know from above, the average D&D project raised more money ($34,115.85) than the average non-D&D project ($32,958.42). But the difference between these averages is only $1,157.43 (not that much considering there are projects raising millions of dollars). So what does the t-test say?

When comparing the average of 353 D&D projects to 675 non-D&D projects, the p-value (chance that the observed difference between the populations could have occurred by chance) was 0.9013. This means that although the observed difference in the average money raised by D&D vs non-D&D projects was not 0, there is about a 90% chance that the difference in money raised is essentially the product of random chance. Statistically speaking, there is no evidence that projects using D&D are more successful than non-D&D projects, and vice versa.

Compare this to the analysis of AI vs non-AI campaigns. The average project using AI raised considerably less money ($6,771.98) than projects that didn't use AI ($39,329.18). But is this difference significant?

Yes! When comparing the average of 189 AI projects vs 841 non-AI projects, the p-value was 0.000001774 (a 0.0001774% chance that the observed difference occurred by chance). The difference in average money raised between AI and non-AI projects is significant. However, we cannot say definitively that the use of AI is the driving force behind this significant result. The correlation is strong, but there could be any number of confounding variables associated with the use of AI. For example, use of AI could heavily correlate with lower quality products or poorly advertised campaigns or lazy ideas or lack of originality...the list goes on. If this was the case (and there's basically no way to test this), then it's not technically the use of AI itself that is driving the difference.

D&D vs Non-D&D Projects (Medians)

One metric we can use to further examine the differences here is looking at the median amount of money raised. Whereas averages are heavily impacted by outliers (looking at you MCDM RPG), median values can give us a good idea of the distribution of money raised by various categories of campaigns.

• January
• All: $4,016.43
• D&D: $3,367.38
• Non-D&D: $5,157.00
• AI: $3,805.52
• February
• All: $3,875.61
• D&D: $3,031.00
• Non-D&D: $3,978.00
• AI: $3,701.65
• March
• All: $3,025.60
• D&D: $2,592.03
• Non-D&D: $3,334.27
• AI: $1,669.64
• April
• All: $5,425.73
• D&D: $3,481.15
• Non-D&D: $2,364.00
• AI: $3,425.78
• May
• All: $7,055.91
• D&D: $8,868.50
• Non-D&D: $6,199.92
• AI: $3,779.22
• June
• All: $4,342.50
• D&D: $4,140.50
• Non-D&D: $4,342.50
• AI: $3,332.00
• Total
• All: $4,120.80
• D&D: $3,433.30
• Non-D&D: $4,516.00
• AI: $3,262.55

Now these data come with a BIG caveat: it's very difficult to compare medians between groups. There are no statistical tests to say whether the difference is statistically significant, and you could have samples with exactly the same median that have wildly different qualities that are hugely relevant (variance, size, average, mode, etc). That said, there are interesting things to be said when comparing the median and average of the same population.

For example, the medians of all categories of projects across all months are considerably lower than the averages. This tells us that even though the average amount of money raised is fairly high, that metric is consistently being pulled up by a few high-earning projects. In theory, an average should be right in the middle of the population, but here we're seeing that when you count the (for example) 73rd project in April when the projects are all arrayed from least to most money raised, that project raised $5,425.73, almost nine times less than the average of $45,029.82. Since this pattern is consistent across all timeframes and categories, this tells us that the majority of projects earn far less money than the average would suggest.

I don't know a way to verify this, but I suspect that this is part of the reason that there isn't a significant difference in money raised by D&D vs non-D&D projects: the average just doesn't represent a strong middle of the actual population. This is somewhat borne out by the median money raised by D&D ($3,433.30) vs non-D&D ($4,516.00) projects. Even though there are fewer D&D projects overall, they have a larger proportion of projects that raise fairly little money and potentially a larger proportion of projects that raise a lot of money.

On that subject, stay tuned for part 3 where I'll delve into those projects that make a lot of money, along with some random for-fun statistics.