Placeholder Image

字幕表 動画を再生する

  • This is a brilliant tweet.

  • But I don't want you to pay attention to the tweet.

  • It's good, sure, but I want you to watch the numbers that are underneath it.

  • That's a screen recording,

  • and the numbers are going up and down, all over the place.

  • They should steadily rise, but they don't.

  • There aren't that many people tapping 'like' by mistake and then taking it back.

  • So why can't Twitter just count?

  • You'll see examples like this all over the place.

  • On YouTube, subscriber and view counts sometimes rise and drop seemingly at random,

  • or they change depending on which device you're checking on.

  • Computers should be good at counting, right?

  • They're basically just overgrown calculators.

  • This video that you're watching,

  • whether it's on a tiny little phone screen or on a massive desktop display,

  • it is all just the result of huge amounts of math that turns

  • a compressed stream of binary numbers into amounts of electricity

  • that get sent to either a grid of coloured pixels or a speaker,

  • all in perfect time.

  • Just counting should be easy.

  • But sometimes it seems to fall apart.

  • And that's usually when there's a big, complicated system

  • with lots of inputs and outputs,

  • when something has to be done at scale.

  • Scaling makes things difficult. And to explain why,

  • we have to talk about race conditions, caching, and eventual consistency.

  • All the code that I've talked about in The Basics so far has been single-threaded,

  • because, well, we're talking about the basics.

  • Single-threaded means that it looks like a set of instructions

  • that the computer steps through one after the other.

  • It starts at the top, it works its way through, ignoring everything else,

  • and at the end it has Done A Thing.

  • Which is fine, as long as that's the only thread,

  • the only thing that the computer's doing,

  • and that it's the only computer doing it.

  • Fine for old machines like this,

  • but for complicated, modern systems, that's never going to be the case.

  • Most web sites are, at their heart, just a fancy front end to a database.

  • YouTube is a database of videos and comments.

  • Twitter is a database of small messages.

  • Your phone company's billing site is a database of customers and bank accounts.

  • But the trouble is that a single computer holding a single database can only deal with

  • so much input at once.

  • Receiving a request, understanding it, making the change, and sending the response back:

  • all of those take time,

  • so there are only so many requests that can fit in each second.

  • And if you try and handle multiple requests at once,

  • there are subtle problems that can show up.

  • Let's say that YouTube wants to count one view of a video.

  • It just has the job of adding one to the view count.

  • Which seems really simple, but it's actually three separate smaller jobs.

  • You have to read the view count,

  • you have to add one to it,

  • and then you have to write that view count back into the database.

  • If two requests come along very close to each other,

  • and they're assigned to separate threads,

  • it is entirely possible that the second thread

  • could read the view count

  • while the first thread is still doing its calculation.

  • And yeah, that's a really simple calculation, it's just adding one,

  • but it still takes a few ticks of a processor.

  • So both of those write processes would put the same number back into the database,

  • and we've missed a view.

  • On popular videos, there'll be collisions like that all the time.

  • Worst case, you've got ten or a hundred of those requests all coming in at once,

  • and one gets stuck for a while for some reason.

  • It'll still add just one to the original number that it read,

  • and then, much later,

  • it'll finally write its result back into the database.

  • And we've lost any number of views.

  • In early databases, having updates that collided like that could corrupt the entire system,

  • but these days things will generally at least keep working,

  • even if they're not quite accurate.

  • And given that YouTube has to work out not just views,

  • but ad revenue and money,

  • it has got to be accurate.

  • Anyway, that's a basic race condition:

  • when the code's trying to do two or more things at once,

  • and the result changes depending on the order they occur in,

  • an order that you cannot control.

  • One solution is to put all the requests in a queue,

  • and refuse to answer any requests until the previous one is completed.

  • That's how that single-threaded, single-computer programming works.

  • It's how these old machines work.

  • Until the code finishes its task and says "okay, I'm ready for more now",

  • it just doesn't accept anything else.

  • Fine for simple stuff, does not scale up.

  • A million-strong queue to watch a YouTube video doesn't sound like a great user experience.

  • But that still happens somewhere, for things like buying tickets to a show,

  • where it'd be an extremely bad idea to accidentally sell the same seat to two people.

  • Those databases have to be 100% consistent, so for big shows,

  • ticket sites will sometimes start a queue,

  • and limit the number of people accessing the booking site at once.

  • If you absolutely must count everything accurately, in real time, that's the best approach.

  • But for sites dealing with Big Data, like YouTube and Twitter,

  • there is a different solution called eventual consistency.

  • They have lots of servers all over the world,

  • and rather than reporting every view or every retweet right away,

  • each individual server will keep its own count,

  • bundle up all the viewcounts and statistics that it's dealing with,

  • and just it will just update the central system when there's time to do so.

  • Updates doesn't have to be hours apart,

  • they can just be minutes or even just seconds,

  • but having a few bundled updates that can be queued and dealt with individually

  • is a lot easier on the central system

  • than having millions of requests all being shouted at once.

  • Actually, for something on YouTube's scale,

  • that central database won't just be one computer:

  • it'll be several, and they'll all be keeping each other up to date,

  • but that is a mess we really don't want to get into right now.

  • Eventual consistency isn't right for everything.

  • On YouTube, if you're updating something like the privacy settings of a video,

  • it's important that it's updated immediately everywhere.

  • But compared to views, likes and comments, that's a really rare thing to happen,

  • so it's OK to stop everything, put everything else on hold,

  • spend some time sorting out that important change, and come back later.

  • But views and comments, they can wait for a little while.

  • Just tell the servers around the world to write them down somewhere, keep a log,

  • then every few seconds, or minutes, or maybe even hours for some places,

  • those systems can run through their logs,

  • do the calculations and update the central system once everyone has time.

  • All that explains why viewcounts and subscriber counts lag sometimes on YouTube,

  • why it can take a while to get the numbers sorted out in the end,

  • but it doesn't explain the up-and-down numbers you saw at the start in that tweet.

  • That's down to another thing: caching.

  • It's not just writing into the database that's bundled up. Reading is too.

  • If you have thousands of people requesting the same thing,

  • it really doesn't make sense to have them all hit the central system

  • and have it do the calculations every single time.

  • So if Twitter are getting 10,000 requests a second for information on that one tweet,

  • which is actually a pretty reasonable amount for them,

  • it'd be ridiculous for the central database to look up all the details and do the numbers every time.

  • So the requests are actually going to a cache,

  • one of thousands, or maybe tens of thousands of caches

  • sitting between the end users and the central system.

  • Each cache looks up the details in the central system once,

  • and then it keeps the details in its memory.

  • For Twitter, each cache might only keep them for a few seconds,

  • so it feels live but isn't actually.

  • But it means only a tiny fraction of that huge amount of traffic

  • actually has to bother the central database:

  • the rest comes straight out of memory on a system that is built

  • just for serving those requests,

  • which is orders of magnitude faster.

  • And if there's a sudden spike in traffic,

  • Twitter can just spin up some more cache servers,

  • put them into the pool that's answering everyone's requests,

  • and it all just keeps working without any worry for the database.

  • But each of those caches will pull that information at a slightly different time,

  • all out of sync with each other.

  • When your request comes in, it's routed to any of those available caches,

  • and crucially it is not going to be the same one every time.

  • They've all got slightly different answers,

  • and each time you're asking a different one.

  • Eventual consistency means that everything will be sorted out at some point.

  • We won't lose any data, but it might take a while before it's all in place.

  • Sooner or later the flood of retweets will stop, or your viewcount will settle down,

  • and once the dust has settled everything can finally get counted up.

  • But until then: give YouTube and Twitter a little leeway.

  • Counting things accurately is really difficult.

  • Thank you very much to the Centre for Computing History here in Cambridge,

  • who've let me film with all this wonderful old equipment,

  • and to all my proofreading team who made sure my script's right.

This is a brilliant tweet.

字幕と単語

動画の操作 ここで「動画」の調整と「字幕」の表示を設定することができます

B1 中級

コンピュータが時々カウントできない理由 (Why Computers Can't Count Sometimes)

  • 0 0
    林宜悉 に公開 2021 年 01 月 14 日
動画の中の単語