Placeholder Image

字幕表 動画を再生する

  • Hello, and welcome to the second video in Module One

  • of, Working With Data & APIS in JavaScript.

  • Now, we're going to do some real stuff in this video.

  • We did real stuff in the previous video,

  • but the previous video was just about practicing with the fetch

  • API and getting some image files.

  • We weren't really working with data

  • and we weren't doing anything with that data yet.

  • In this video, I want to take a look

  • at this idea of tabular data.

  • There are a lot of different file

  • formats for storing data in a table format, in tabular data.

  • The one that I want to look at in this video,

  • and probably I would think is the most common one, is CSV,

  • or comma-separated value.

  • Meaning the data, the data in the table,

  • is literally separated by commas.

  • The first line of text, after all, this file, the CSV file,

  • is ultimately just a plain text file.

  • It might function as something like a hedero.

  • So it would have the names of the fields

  • of data you're going to have.

  • So it might have something like, Item, comma, Cuteness.

  • So you're going to have a table of things and a cuteness score.

  • So all the rest of the lines would have the actual items

  • and their cuteness scores.

  • So you could have, puppy, comma, 10.

  • Kitten, comma, 10.

  • Because we live in a world where everything has a cuteness

  • curve 10.

  • But I want to do something with real data, data that's

  • out there in the world, that I can

  • grab with the fetch function, load onto my web page,

  • and do something with, for example, graph.

  • And so the data set that I'm going to show you,

  • it comes from NASA, National Aeronautics

  • and Space Administration, in particular from the Goddard

  • Institute for Space Studies.

  • This CSV file includes the combined global average land

  • surface air and sea surface water temperature

  • from 1880 all the way to present.

  • It's stored in sort of a funny way.

  • The values that are actually in the data set

  • are the difference from the mean temperature.

  • And what do I mean by the mean temperature?

  • Well, I mean the average temperature.

  • Well, what's the average temperature?

  • Well, it so happens that there is the average world

  • temperature, as recorded from 1951 to 1980, which also

  • recorded an average by NASA, the Earth Observatory website,

  • at 14 degrees Celsius.

  • So the data in this CSV file is the difference from the mean,

  • from 14 degrees Celsius, from combined land surface

  • air and sea surface water temperature from 1880

  • to present.

  • So I want to load that CSV file, parse it, graph it,

  • and we're done.

  • I'm going to do this in two parts.

  • The first part, that you're watching right now,

  • is just loading the CSV file, parsing it.

  • I want to be able to see it, maybe

  • as a console log in the browser, in the Chrome Developer Tools.

  • And then once I see I have the data there,

  • then I want to try to graph it.

  • I'm going to graph it using a particular JavaScript library,

  • called Chart.js.

  • I'll talk about some other ways that you can choose

  • to graph stuff, then, as well.

  • If you want to follow along with coding,

  • I'm about to start coding, you're

  • first going to want to grab that CSV file

  • and have it stored locally on your computer.

  • So it's a pretty easy process.

  • You just want to go to data.giss.nasa.gov/gistemp.

  • The URL's here and in the video's description.

  • And then you want to scroll all the way down and find the place

  • on the web page that says, Tables of Global

  • and Hemispheric Monthly Means and Zonal Annual Means.

  • So, there are actually a ton of different data sets on this web

  • page, and you might explore them,

  • and perhaps as an exercise try doing

  • graphing a different data set on this web page.

  • But the one that I'm using is the last entry

  • on that section called, Zonal Annual Means,

  • from 1880 to present.

  • And I'm using the CSV file format.

  • You'll notice there's also a TXT file format, that's probably

  • tab delimited so that the sort of data records

  • have tabs in between them instead of commas.

  • Again, there's a variety of different formats,

  • but the CSV is the one that I'm going to use.

  • Time to start coding.

  • So let's check in and see if you--

  • if you want to follow along, let's check in

  • and see if you have exactly what I have.

  • So what I have is some boiler plate HTML.

  • It's just a plain index of HTML file

  • with a title, a head, a body, and an empty script tag.

  • I've got links to where the data is coming from,

  • just to make sure I'm referencing and crediting

  • properly in my code.

  • And then I also have that CSV file itself.

  • So, yeah, I'm in Visual Studio Code.

  • And you can see, there's my index of HTML

  • and there's my CSV file in the same local directory.

  • But you might be using a different text

  • editor or a different environment, all of this

  • will work as long as you have your HTML file

  • and your CSV file.

  • Let's take a look at that CSV file.

  • So here's the CSV file.

  • You can see that there are a number of columns: Year; Glob,

  • which I assume stands for global; Nhem ,

  • northern hemisphere; et cetera, et cetera.

  • And then you can see that the columns of data each

  • being separated by commas.

  • So this isn't really meant to be human readable.

  • There are ways of viewing a CSV that's more human readable.

  • For example, here's how that same data

  • looks in spreadsheet format.

  • You might notice here, however, that it is colored.

  • Each column has a different color.

  • This is because I'm using a Visual Studio Code

  • extension called Rainbow CSV, which it says if it

  • was like tailor made for me.

  • And I'll include a link in this video's scripture

  • if you want to install that extension as well so you

  • can have things color coded.

  • Another thing I really like to do

  • when I'm working with a data set for the first time,

  • is I like to give myself a test file that

  • has much less stuff in it.

  • Because if I want a console, log, and check stuff,

  • sometimes this big file-- this isn't that big of a file,

  • it's just 1880 to 2018.

  • But, potentially, I could have like a really, really big data

  • file.

  • So something I'm going to do is, I'm

  • going to just quickly do a Save As and call this test.csv.

  • And so now you can see that I have a separate test.csv file.

  • And I'm going to just leave two years in there.

  • So I'm going to scroll all the way down and delete everything.

  • And now I have a CSV file that just has three rows in it:

  • the header row and then the data for 1880 and the data for 1881.

  • So I'm going to work with this first.

  • And once I have the parsing and everything

  • I want to do working properly, then I'll load the real data.

  • So the first step is exactly what you might think,

  • fetch test.csv.

  • Let's write that code.

  • I'm going to write, fetch test.csv.

  • And then remember, fetch returns a promise,

  • and I can handle the resolution of that promise

  • when it is finished loading the data with dot vin

  • and any errors with dot catch.

  • But I prefer to use the await and async syntax,

  • so I'm actually going to put this in a function called,

  • get data.

  • I might think of a different name for that later.

  • And I'm going to say the response equals

  • a weight fetch test.csv.

  • So I'm rating a function-- oh, and this function,

  • I almost forgot, has to have the keyword async because it's

  • an asynchronous function that's making asynchronous calls

  • with the await keyword.

  • So the response equals await fetch dot test CSV.

  • Now you might remember, on the web

  • fetch API there are a variety of kinds of data streams

  • that might come in.

  • There is a blob, there's a JSON, there's an array buffer,

  • there's text.

  • And this is what I want to actually use, raw text.

  • Even though it's tabular data in CSV format,

  • I'm going to do the parsing of it manually in my own code.

  • So I just want to receive it as text.

  • And that means--

  • I'm going to say--

  • I'm just going to say table equal--

  • maybe I'll just call this all data,

  • equals await a rate response dot text.

  • So let's console log that data.

  • And let's call the function, get data here.

  • And then let's go and see this running actually

  • in the browser.

  • And here it is.

  • And you can see, there we go, the data

  • has been logged to the console.

  • Now, ultimately, here there are a variety

  • of JavaScript libraries that will parse a CSV for you.

  • And by I mean, parse, I mean figure out

  • where all the commas are and break up the data

  • and put it into objects and make it usable for you.

  • D3, which is a JavaScript library for data visualization,

  • has a parser in it. p5 js, which is a JavaScript library

  • that I use a lot on this channel,

  • has a load table function, which will actually

  • parse the CSV for you.

  • And there are many others.

  • So I'll include some links to those in the video description.

  • But I think it's a useful exercise right now.

  • It's a simple enough data for us to do the parsing manually

  • with the split function.

  • What?

  • What split function?

  • What are you talking about?

  • So the JavaScript string class, any time

  • you have a piece of text in a variable in JavaScript

  • it's a string object, has a function called split.

  • And that function allows you to take any arbitrary text

  • and split it up into different elements of an array.

  • And that's basically what we want to do.

  • We want to split up all the rows, and then each row,

  • we want to split up all the columns.

  • The split function requires a single argument, a separator

  • or otherwise known as a delimiter.

  • And in this case, we have two kinds of delimiters.

  • For each row, the delimiter, the thing

  • that differentiates one row from another, is a line break.

  • So first let's call split with line break.

  • Going to my code, I can say, and I'm going to call these rows,

  • the rows equals data dot split.

  • And I'm going to split by backslash n.

  • So backslash n is an escape character sequence

  • that indicates a line break or new line.

  • Depending on your file format, you

  • might need a slash r, also which is like a carriage return.

  • You can also use something called a regular expression

  • here.

  • This should also work.

  • Instead of in quotes, if I have forward slashes,

  • the delimiter is a regular expression.

  • What's a regular expression.

  • So that's beyond the scope of what

  • we're doing in this particular video,

  • but I think expressions are so useful when doing string

  • parsing, that I will also link in this video description

  • to a whole series of videos that I have about that.

  • But for now, just the backslash n in single quotes

  • should do for us.

  • So I'm going to say console.log rows,

  • just to make sure that works.

  • And it does.

  • So we can see here, this is the raw text.

  • And now this is split into an array with three elements,

  • each element is one line in that array.

  • And one thing, though, we don't actually need to first row.