| Study parameter | Commitment |
|---|---|
| Population | Studio catalogues of 120 most-streamed rap artists of 2016 to 2026 |
| Measures | Unique word ratio, MATTR, rhyme density, syllables per bar |
| Corpus source | MusicBrainz for artist and release metadata |
| Lyric handling | Computed on, never stored or republished. Derived metrics only |
| Excluded | Features, skits, live versions, remixes, posthumous compilations |
| Released as | CC-BY 4.0 dataset plus method, archived with a DOI |
| Refresh | Quarterly, with a dated changelog per revision |
Why lexical density is worth measuring at all
Lexical density is the closest thing rap criticism has to a falsifiable claim. Arguments about who is a better lyricist collapse into taste almost immediately, but "how many distinct words does this artist actually use across their catalogue" is a question with an answer, and the answer can be checked by anyone willing to recount it.
The format has precedent. The Pudding's vocabulary study of hip-hop established both the appetite and the method's limits, and it remains the reference point every follow-up is measured against. This study is not an attempt to replace it. It is an attempt to run the same class of measurement on a current population, with the corpus rules published in advance and the raw output released openly.
What the four measures actually capture
The four measures each capture a different property, and they are reported separately rather than combined into a single score. A composite would hide exactly the disagreements that make the data useful.
- Unique word ratio counts distinct words against total words across an artist's studio catalogue. It is the most intuitive measure and the most sensitive to catalogue length, which is why it is never reported alone.
- Moving-average type-token ratio (MATTR) computes the same ratio inside a sliding 500-word window and averages the result. It exists because raw type-token ratio falls mechanically as a text gets longer, so comparing a two-album artist to a twelve-album artist on raw ratio measures catalogue size, not vocabulary.
- Rhyme density counts rhymed syllables against total syllables. It captures a property that vocabulary counts miss entirely: an artist can rank low on unique words and extremely high on internal rhyme.
- Syllables per bar measures delivery density rather than vocabulary, and is included as a control. If it correlates strongly with the vocabulary measures, that is itself a finding about what the vocabulary measures are really tracking.
How the corpus is built, and what is deliberately thrown away
The corpus is built from MusicBrainz artist and release metadata rather than from a streaming platform's catalogue, because MusicBrainz is openly licensed, community-audited, and stable enough to reproduce a corpus from a stated snapshot date. Population selection uses published streaming-volume figures, and the exact source and snapshot date ship in the method note with the results rather than being asserted here, because a source named in advance and quietly swapped later is the failure this pre-registration exists to prevent.
Exclusions are committed here rather than decided once the numbers look wrong. Guest features are excluded because attribution of a verse to an artist inside another artist's release is inconsistent in metadata. Skits are excluded because they are speech, not lyrics. Live versions and remixes are excluded because they duplicate text already counted. Posthumous compilations are excluded because their contents are editorial decisions made by estates rather than by the artist.
Every one of those exclusions changes the result, some of them substantially. That is precisely why the rule is published before the run.
How lyrics are handled without republishing them
Lyrics are copyrighted works, and this study never republishes them. Lyric text is retrieved through the Genius API, computed on in memory, and discarded. What ships is derived metrics: counts, ratios and densities. No lyric line appears in the dataset, the article, or the archive.
This constraint is the reason the dataset can be openly licensed at all. A corpus containing the underlying text could not be released under CC-BY 4.0, because the rights to that text are not ours to grant. Derived statistical measures are a different object. The restriction is not a compromise on openness, it is what makes the openness lawful.
What would prove this measure worthless
Pre-committing the failure conditions is the point of publishing a method before results. Three outcomes would tell us this measure does not do what it claims, and each is stated now so that neither we nor a reader can rationalise it later:
Pre-committed falsifiers
- Catalogue-length dominance. If MATTR still correlates with catalogue size above r = 0.5 after windowing, the window is not doing its job and the rankings are measuring discography length. We report that and the measure does not ship.
- Control collapse. If syllables per bar correlates with the vocabulary measures above r = 0.7, the study is measuring delivery speed wearing a vocabulary costume. That result publishes as the finding.
- Transcription instability. If re-running the same artist against an independently sourced transcript shifts unique word ratio by more than 5%, the measure is dominated by transcription noise rather than by the artist, and the confidence interval swallows the ranking.
Any of those three outcomes gets published with the same prominence as a clean result. A method that only reports itself when it works is not a method.
Limitations, written before the results
Limitations are listed here rather than appended after publication, when the temptation is to phrase them so they do not undercut the headline.
Transcription quality is the largest source of error and it is not evenly distributed: heavily annotated mainstream catalogues are transcribed far more accurately than regional or independent ones, which means measurement error correlates with fame. Ad libs are inconsistently transcribed and materially affect syllable counts. Rhyme detection is grapheme-based and will systematically undercount slant rhyme and regional pronunciation, which is a real bias against specific regional traditions rather than a neutral noise floor. And catalogue boundaries are an editorial judgement no matter how carefully the exclusion rules are written.
How to reuse the data when it publishes
The dataset publishes under CC-BY 4.0 with a persistent identifier through Zenodo, so a citation resolves permanently rather than to a URL that may move. Commercial reuse is permitted. Attribution and a link back to this study is the entire requirement, and no permission request is necessary.
The method note, the exclusion rules and the snapshot date ship in the same archive, so a third party can rebuild the corpus and disagree with the result on its merits. Pre-registration norms are borrowed here from the Open Science Framework, where committing a method before collection is ordinary practice.
Why publish a method before you have any results?
Because a method published after results can always be shaped to fit them, and no reader can tell the difference. Committing the corpus rules and the falsifiers in advance is the only way a music statistic can be trusted by someone who was not in the room.
Will you release the lyric corpus itself?
No. Lyrics are copyrighted works. They are computed on and discarded, and only derived metrics are released, which is both lawful and the reason the dataset can carry an open licence.
Why 120 artists and not more?
120 is the point where the population is defensible from streaming-volume data without the tail becoming dominated by artists whose catalogues are too sparsely transcribed to measure reliably. If the transcription-quality analysis shows the tail is unusable, the reported n drops and we say so.
Does a higher score mean a better rapper?
No, and the study will not claim it does. It measures four specific properties of a text. Anyone using it as a ranking of skill is making an argument the data does not support.
When does it publish?
When the corpus completes and the falsifier checks have run. There is no publication date being worked toward, because a deadline is how a method quietly becomes negotiable.
Formal literature citations for the type-token and rhyme-density measures publish with the results rather than here, because a reference list assembled to decorate a method page is worth less than one assembled to support a finding. See the methodology for the standard every Overtone study is held to, and open datasets for the release terms.