The Zipf Mystery

15 sep 2015
18 917 712 Áhorf

The of and to. A in is I. That it, for you, was with on. As have ... but be they.
RELATED LINKS AND SOURCES BELOW!
twitter.com/tweetsauce
instagram.com/electricpants
WordCount.org www.wordcount.org/
How many days have you been alive? www.beatcanvas.com/daysalive.asp
random letter generator: www.dave-reed.com/Nifty/randSeq.html
Dictionary of Obscure Sorrows: ispast.info
Word frequency resources:
[lemmatized] en.wikipedia.org/wiki/Most_common_words_in_English
www.uow.edu.au/~dlee/corpora.htm
www.wordfrequency.info
www.anc.org/data/anc-second-release/frequency-data/
www.titania.bham.ac.uk/docs/
www.kilgarriff.co.uk/bnc-readme.html#raw
en.wiktionary.org/wiki/Wiktionary:Frequency_lists
ucrel.lancs.ac.uk/bncfreq/
[PDF] www.wordfrequency.info/files/entries.pdf
[combined Wikipedia and Gutenberg] www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/
corpus.byu.edu/coca/files/100k_samples.txt
corpus.byu.edu/
corpus.leeds.ac.uk/list.html
books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr=
www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html
Great Zipf's law papers:
colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf
www.ling.upenn.edu/~ycharles/sign708.pdf
arxiv.org/pdf/cond-mat/0412004.pdf
www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf
Zipf’s law articles and discussions:
www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/
io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?.Ej9760cOTJCPS_Bq4mjoww.0
plus.maths.org/content/os/latestnews/may-aug08/food/index
judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em
plus.maths.org/content/mystery-zipf?src=aop
www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons
en.wikipedia.org/wiki/Zipf%27s_law
books.google.co.uk/books?id=f8GrzlnMSm8C&pg=PA62&redir_esc=y#v=onepage&q&f=false
arxiv.org/pdf/0802.4393v1.pdf
www.pnas.org/content/108/9/3526.full
lewisdartnell.com/language_page.htm
wugology.com/zipfs-law/
other Zipf’s law PDFs
ftp.iza.org/dp3928.pdf
arxiv.org/pdf/1402.2965.pdf
arxiv.org/pdf/1104.3199.pdf
www.lel.ed.ac.uk/~jim/zipfjrh.pdf
www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1
polymer.bu.edu/hes/articles/pgs02a.pdf
in untranslated language: arxiv.org/pdf/0808.2904.pdf
pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf
www.hpl.hp.com/research/idl/papers/ranking/ranking.html
statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf
arxiv.org/pdf/1310.0448v3.pdf
www.kornai.com/Papers/glotto5.pdf
Zipf’s law slides:
www.slideshare.net/guest9fc47a/nlp-new-words
Pareto Principle and related ‘laws’:
www.squawkpoint.com/2013/03/pareto-principle/
billyshall.com/blog/post/paretos-principle
en.wikipedia.org/wiki/Pareto_principle
Random typing and Zipf:
www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html
health 80/20: archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html
Principle of least effort:
en.wikipedia.org/wiki/Principle_of_least_effort
en.wikipedia.org/wiki/Satisficing
www.pnas.org/content/100/3/788.full.pdf [PDF]
csiss.org/classics/content/99
self organized criticality:
journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full
Hapax Legomenon:
campus.albion.edu/english/2011/02/15/hapax-legomenon/
www.dailywritingtips.com/is-that-a-hapax-legomenon/
en.wikipedia.org/wiki/Hapax_legomenon
[PDF] www.aclweb.org/anthology/J10-4003
www.wired.com/2012/01/hapax-legomena-and-zipfs-law/
oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1
oed.hertford.ox.ac.uk/main/content/view/36/166/index.html
Learning curve: en.wikipedia.org/wiki/Learning_curve
Forgetting curve:
www.trainingindustry.com/wiki/entries/forgetting-curve.aspx
en.wikipedia.org/wiki/Forgetting_curve
Experience curve effects: en.wikipedia.org/wiki/Experience_curve_effects
Forgetting
and zipf's law: act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf
public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf
marshalljonesjr.com/youll-remember-less-than-001-of-your-life/
en.wikipedia.org/wiki/Forgetting
www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/
music from:
ispast.info
www.audionetwork.com

Ummæli
  • "Zipf-y"

    Will It Snap?Will It Snap?Klukkustund síðan
  • Vsauce Say "I want" and then everything after it without (The)

    Junovah HunziilJunovah HunziilKlukkustund síðan
  • What a video!!!

    Complex GodComplex God2 klukkustundum síðan
  • Video must be old. The most common word now is literally :p

    Scott KeselowskiScott Keselowski3 klukkustundum síðan
  • everyone use the second most used word as much as possible

    fewbronze gamesfewbronze games4 klukkustundum síðan
  • This can't be right ....

    Itchy Eyes McRealizeItchy Eyes McRealize4 klukkustundum síðan
  • I know use quizatiously once at least a day I no speel

    Joe BlackmoreJoe Blackmore6 klukkustundum síðan
  • Well order is the last thing that you'd think chaos would do, so thats exactly what it did

    The Failed_safeThe Failed_safe8 klukkustundum síðan
  • Have you forgotten this video now that 5 years have passed?

    Hugo Le RoyHugo Le Roy9 klukkustundum síðan
  • He got us in the first half, not gonna lie.

    SoftAlexandra3 PlaysSoftAlexandra3 Plays10 klukkustundum síðan
  • 0:26 This is a really very good Poem.

    Helpful SysopsHelpful Sysops13 klukkustundum síðan
  • "The" dont exist in my language. You change the word instead that wold be after "the".

    RubbeRubbe16 klukkustundum síðan
  • This is like the Fibbonaci Sequence.

    Ethereal Sky DaddyEthereal Sky Daddy19 klukkustundum síðan
  • What if we tend to use more words than others because some might feel repetitive? If someone puts "the" 30 times in an essay it will certainly look better than if they put "furthermore" also 30 times in it.

    Sebastian SolisSebastian Solis22 klukkustundum síðan
  • Your science is flawed as only a few letters on their own are actually words. F or Q are not words for example, they are letters.

    tdchewytdchewy23 klukkustundum síðan
  • After the intro his percentage when the up to 7%

    [GD] Goose[GD] GooseDegi Síðan síðan
  • Im scared of this \ line now

    Cattu DogguCattu DogguDegi Síðan síðan
  • 14:08 imagine this in a bernie sanders voice LOL

    Mag MagnetMag MagnetDegi Síðan síðan
  • Okay but Vilfredo Pareto has to be the coolest name ever

    JayleighJayleighDegi Síðan síðan
  • every time micheal says the word "the" I go: "Micheal stop! STOP! You're killing me!"

    Double A-AADouble A-AADegi Síðan síðan
  • me: how is this possible? Vsause: yes

    Eric YoungEric YoungDegi Síðan síðan
  • V-sauce. Really? "Its like if you take a snowball and roll it down a snowy hill." Yes my genius friend, the snowball effect.

    Benjamin CoffmanBenjamin CoffmanDegi Síðan síðan
    • Still love you Michael :p

      Benjamin CoffmanBenjamin CoffmanDegi Síðan síðan
  • Haha the

    Liam BinsfeldLiam BinsfeldDegi Síðan síðan
  • So, quizzaciously is a fun word, but can it beat fabaceous (Adv. of the nature of the bean)?

    Eye Master 3.14Eye Master 3.14Degi Síðan síðan
  • Quizzaciously

    InsteadImAwakeInsteadImAwakeDegi Síðan síðan
  • That intro is what makes time continue

    qwzerty_qwzerty_2 dögum síðan
  • The better quart enzymatically whistle because flugelhorn likely shade under a onerous care. boring, mushy link

    Bernice GardnerBernice Gardner2 dögum síðan
  • 11:27 top ten anime plot twists

    Ultimate FlanUltimate Flan2 dögum síðan
  • This guy is extraordinary

    afeelfortheflow with Manicafeelfortheflow with Manic2 dögum síðan
  • AAAAAAAA! Vsauce is making me interested in math! NOOOOO What do I do lol

    Magic BeansMagic Beans2 dögum síðan
  • so if i study onlky 20% of most used words & grammar i can learn 80% of a language 🤑

    CieloCielo2 dögum síðan
  • This is like almost actual proof we are in a simulation

    A1 HelixA1 Helix2 dögum síðan
  • Since there are an infinite causes and effect pairs, I believe every arbitrary combination like 30/70 or 50/50 will also have infinite occurences (there may not be anything special about 20/80). If someone shows that the popularity or frequency of a particular combination, say 20/80, is more compared to the rest of the possibilities, then it would be something weird. Listing a 100 examples which follows 20/80 principle from an infinite more total examples does not lead me to believe that this combination is special. Nice video though, as always 😉.

    Vivek SaraswatVivek Saraswat2 dögum síðan
  • We broke the law!!! Only 9 percent of plastic is recycled 91 percent is in landfills or is litter.

    Thomas BeasleyThomas Beasley3 dögum síðan
  • 80% of my life is boring but 20% is more boring.

    {Davidcool}{Davidcool}3 dögum síðan
  • We have the Illuminati all figured out now 😏. They thought they could pull the Zipf over our eyes. Not so much anymore.

    THE JACK HQTHE JACK HQ3 dögum síðan
  • 4:23 George Zipf é a cara do jornalista Augusto Nunes.

    Hugo PetrarolhaHugo Petrarolha3 dögum síðan
  • Vilfredo Pareto

    Eli PhelpsEli Phelps3 dögum síðan
  • i didnt understand any of that

    Emlar TubbsEmlar Tubbs3 dögum síðan
  • Of all the stuff I have 20% is useful, 80% is junk..

    RainManRainMan4 dögum síðan
  • the word "um" is used more than "the

    DynaBloxDynaBlox4 dögum síðan
  • 80% of my code sucks and 20% of my code sucks whats the problem then

    Harvey YFBYHarvey YFBY4 dögum síðan
  • It made me sure that I am living in a stimulation: (

    yash Payalyash Payal4 dögum síðan
  • 80% of micheals hair is on 20% of his head

    Cesar ArizpeCesar Arizpe4 dögum síðan
  • maybe because we all run in one ai program

    GodGod4 dögum síðan
  • How can you not translate the unknown language, all ya gotta do is figure out what's used the most then that must be the word "the" and so on.

    Carla MaciasCarla Macias4 dögum síðan
  • All this tells me is that life is actually just a lot of annoying math problems I don't want to do

    jisdaboss1jisdaboss14 dögum síðan
  • Why do I only watch vsauce at 5 in the morning

    That GuyThat Guy5 dögum síðan
  • Zipfs law seems like an Easter egg the creators of the world put in when they were designing us, in the simulation, we're just part of a simulation created by other beings and they made things like zipfs law intentionally for us to eventually find. Imagine the things we don't yet know, or even better imagine the things we think we know but are just based off of speculation

    Jonathan SheddJonathan Shedd5 dögum síðan
    • Speculation? Oh, you mean like speculation as in there must be a creator

      BFB-DanceySteveBFB-DanceySteve3 dögum síðan
  • He is so quizacious

    Alex WallAlex Wall5 dögum síðan
  • Someone should do this for the 3 holly books. Just for fun.

    Tarik TezcanTarik Tezcan5 dögum síðan
  • Cycloid ... 🙌🏽

    Dead ManDead Man5 dögum síðan
  • @Vsauce

    Asif MolviAsif Molvi5 dögum síðan
  • @vsauce please make a video of benfords law

    Asif MolviAsif Molvi5 dögum síðan
  • I got through about half of this video.

    xxxgammaxxxxxxgammaxxx5 dögum síðan
  • That was until this new generation. Now it seems that every other word is "like".

    billy watkinsbilly watkins6 dögum síðan
  • 80% of my life has been wasted on 20% success.

    T. L.T. L.6 dögum síðan
  • Zipf's law treats statistics quizzatiously.

    DJ matexxDJ matexx6 dögum síðan
  • 80 percent of diamonds are mined by 20 percent of players

    Parallax WRParallax WR6 dögum síðan
  • I vote that we all copy paste the word sauce over and over and throw off the whole Zipf thing off. It will be Anarchy. Cats living with dogs. I’m an evil genius. sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce sauce I said off twice just for fun.

    The Tattoo Trunk Show with Scott HillThe Tattoo Trunk Show with Scott Hill6 dögum síðan
  • This just proves whoever is running the simulation got lazy on some details lol

    Paige BellPaige Bell6 dögum síðan
  • I almost clicked on another video when he did the fake ending.

    Axel de WaterAxel de Water6 dögum síðan
  • I'm curious what effect the application of anti-trust laws has on these distributions. We know that when we apply these laws it tends to improve the economy and the quality of life for the customers of these companies. But if businesses tend toward this equilibrium of unfairness then a more heavy handed a approach might be warranted.

    SandyRiverBlueSandyRiverBlue6 dögum síðan
  • "Even so, they have made me" I love this sentence. I lost my grandfather almost a month ago and he was the one who was truly passionate about studying. He had inspired me to continue my studies after high school and I am still passionate about studying. I've been thinking a lot of him during this month, but I realized I was and am not able to remember much things about him as I would like to. Probably this link doesn't make sense, but that sentence Michael said here makes me feel better about it. Thank you.

    WallijacaneroWallijacanero6 dögum síðan
  • The decisive development comparably tease because pail similarly return round a lewd bead. scandalous, deadpan hardboard

    Ana BurkeAna Burke6 dögum síðan
  • What if the zipfs law thing isn’t a flaw in the world but a flaw in how we record and measure the world

    Bernard chezmond dinglechalkBernard chezmond dinglechalk6 dögum síðan
  • Confused unga bunga

    Jonah MaddocksJonah Maddocks6 dögum síðan
  • Mandarin has no translation for "the" Mikey! They also use use less polite words in common language. Like, you wouldn't ask a waitress, "Could I have the check please?" you would say, "Bring check now!" as far as a literal translation goes. It's not considered rude, that's just how that culture speaks. People learning Mandarin might use formal words like please and it will usually make a native Mandarin speaker laugh or at least smile because it seems odd to them.

    rwfrench66rwfrench666 dögum síðan
  • Can't I ruin this whole thing by going to Wikipedia and typing sauce 100000 times and publish it? If you say that wouldn't be counted what if I just say it in real life 100000 times, then it has appeared so you have to count it... Can this just be completely thrown off?

    Random UserRandom User7 dögum síðan
  • As a nurse I wonder what the most common words I use in the hospital are? Maybe they are still the usual suspects

    Doubledge11Doubledge117 dögum síðan
  • I don't really have an opinion on whether or not we live in a simulation, but I would accept that Zipf's Law is the result of a low-level algorithm trying to simulate randomness.

    PeregrinePeregrine7 dögum síðan
  • the word frequency and rank thing is weird to me cause it seems like that's not even really true?? I've only seen about the first minute of the video but when he shows the Shakespeare graphic at 1:17, the 1/2, 1/3, 1/4 thing doesn't happen at all, it goes 1st "and" (~650), 2nd "the" (~650), 3rd "I" (~575) and trends downward for the rest of it. I agree the graph looks like a 1/x graph, but it's not like as the word rank goes from 1 to 2 it halves. The rest of the graph follows the trend, but the beginning of it (the most interesting part of it) doesn't even really do it

    Smudged OutSmudged Out7 dögum síðan
    • As I'm watching the rest of the video, the other examples still hold up, it's just weird that the first example they chose to show didn't actually show it

      Smudged OutSmudged Out7 dögum síðan
  • Who else remembers their locker number? Mines 16,18,37

    QvQQvQ7 dögum síðan
  • Huh, I can't believe it's been 5 years since this came out. I distinctly remember this video, alongside the "Banach Tarski Paradox" video.

    SchippaiSchippai7 dögum síðan
  • Someone has to count how many times Michael says “the” to figure out the secret number to unlocking the secrets of the universe

    Nathan MotoyamaNathan Motoyama7 dögum síðan
  • 20% of my reindeer do 80% of the flying 🎄🦌

    Santa ReviewsSanta Reviews7 dögum síðan
  • 100% of my problems are caused by one woman so.......

    Kuwaitisnot adeploymentKuwaitisnot adeployment7 dögum síðan
  • brb gonna go put the entire script into a google doc then a google form rip sleep 😀

    Maggie HoughMaggie Hough8 dögum síðan
  • im sorry but how is sauce 5,555 and cheltenham is 5,556 in the great british language index

    joe squirejoe squire8 dögum síðan
  • Are you sure a person can make a sentence without it. Most used word, please.

    Dwayne MairDwayne Mair8 dögum síðan
  • The the the the the the

    ShafiqShafiq8 dögum síðan
  • Thai has no articles, or plurals, or gender pronouns. Oh dear...You should get out more.

    J GarboJ Garbo8 dögum síðan
  • Can some count the number of "the" in the video

    MetaReaper zippidy do daMetaReaper zippidy do da9 dögum síðan
  • Combine this with the Brachistochrone video and one might assume that 80% of the acceleration is done over the first 20% of the distance.

    BergBergBergBerg9 dögum síðan
  • how does this relate to bendford's law?

    VitorVitor9 dögum síðan
  • 0:03-0:13 haha I'm german

    fetterFischfetterFisch10 dögum síðan
  • Ever heard of benforrd's law ? Its the same thing

    Shubham SinghShubham Singh10 dögum síðan
  • QUIZZACIOUSLY!

    Alex B.Alex B.10 dögum síðan
  • Omg wow

    Rebecca RachelRebecca Rachel10 dögum síðan
  • I'd personally say this is one of the most significant videos on the Internet. I spent a couple of months once just looking into this phenomenon and it is very interesting.

    Alpha DeltaAlpha Delta10 dögum síðan
  • The simulation in which we run has some laws in its code like zipfs, some series like fibonacci and then the neural network trained to almost 99.99% put that into everything - it proves we're in a simulation

    yatharth saxenayatharth saxena11 dögum síðan
  • Between infinite periodicals like pi and eulers number, and zipfs law... not too far off thinking someone is simulating reality for the fuck of it

    VaQMNRGVaQMNRG11 dögum síðan
  • Thelongestwordinthe dictionary is: ANTIDISASTABLISHMENTARIANISM;ha! ha!

    Lois Ray McInnisLois Ray McInnis11 dögum síðan
  • proof of the matrix

    joe mamajoe mama11 dögum síðan
  • Answer - more common than word "ask". Opposite to what was said.

    sloma111sloma11111 dögum síðan
  • 80% of Vsauce videos were uploaded in a span of 20% of his youtube journey And as always thanks for reading :)

    random boirandom boi11 dögum síðan
  • 80% of what i remember is 20% of my whole memory

    Leonardo ZilmarLeonardo Zilmar11 dögum síðan
  • man ....so many story ideas i forgot

    Rusty RalstonRusty Ralston11 dögum síðan
  • WAIT. Another thing, 80 percent of the money in america.... is held by..... oh wait, that's the top 1%.

    WhatThuSmellWhatThuSmell11 dögum síðan
  • Michael's Age 34

    Pramod BishtPramod Bisht12 dögum síðan
ISpast