字幕表 動画を再生する
-
Hello, and welcome to the second video in Module One
-
of, Working With Data & APIS in JavaScript.
-
Now, we're going to do some real stuff in this video.
-
We did real stuff in the previous video,
-
but the previous video was just about practicing with the fetch
-
API and getting some image files.
-
We weren't really working with data
-
and we weren't doing anything with that data yet.
-
In this video, I want to take a look
-
at this idea of tabular data.
-
There are a lot of different file
-
formats for storing data in a table format, in tabular data.
-
The one that I want to look at in this video,
-
and probably I would think is the most common one, is CSV,
-
or comma-separated value.
-
Meaning the data, the data in the table,
-
is literally separated by commas.
-
The first line of text, after all, this file, the CSV file,
-
is ultimately just a plain text file.
-
It might function as something like a hedero.
-
So it would have the names of the fields
-
of data you're going to have.
-
So it might have something like, Item, comma, Cuteness.
-
So you're going to have a table of things and a cuteness score.
-
So all the rest of the lines would have the actual items
-
and their cuteness scores.
-
So you could have, puppy, comma, 10.
-
Kitten, comma, 10.
-
Because we live in a world where everything has a cuteness
-
curve 10.
-
But I want to do something with real data, data that's
-
out there in the world, that I can
-
grab with the fetch function, load onto my web page,
-
and do something with, for example, graph.
-
And so the data set that I'm going to show you,
-
it comes from NASA, National Aeronautics
-
and Space Administration, in particular from the Goddard
-
Institute for Space Studies.
-
This CSV file includes the combined global average land
-
surface air and sea surface water temperature
-
from 1880 all the way to present.
-
It's stored in sort of a funny way.
-
The values that are actually in the data set
-
are the difference from the mean temperature.
-
And what do I mean by the mean temperature?
-
Well, I mean the average temperature.
-
Well, what's the average temperature?
-
Well, it so happens that there is the average world
-
temperature, as recorded from 1951 to 1980, which also
-
recorded an average by NASA, the Earth Observatory website,
-
at 14 degrees Celsius.
-
So the data in this CSV file is the difference from the mean,
-
from 14 degrees Celsius, from combined land surface
-
air and sea surface water temperature from 1880
-
to present.
-
So I want to load that CSV file, parse it, graph it,
-
and we're done.
-
I'm going to do this in two parts.
-
The first part, that you're watching right now,
-
is just loading the CSV file, parsing it.
-
I want to be able to see it, maybe
-
as a console log in the browser, in the Chrome Developer Tools.
-
And then once I see I have the data there,
-
then I want to try to graph it.
-
I'm going to graph it using a particular JavaScript library,
-
called Chart.js.
-
I'll talk about some other ways that you can choose
-
to graph stuff, then, as well.
-
If you want to follow along with coding,
-
I'm about to start coding, you're
-
first going to want to grab that CSV file
-
and have it stored locally on your computer.
-
So it's a pretty easy process.
-
You just want to go to data.giss.nasa.gov/gistemp.
-
The URL's here and in the video's description.
-
And then you want to scroll all the way down and find the place
-
on the web page that says, Tables of Global
-
and Hemispheric Monthly Means and Zonal Annual Means.
-
So, there are actually a ton of different data sets on this web
-
page, and you might explore them,
-
and perhaps as an exercise try doing
-
graphing a different data set on this web page.
-
But the one that I'm using is the last entry
-
on that section called, Zonal Annual Means,
-
from 1880 to present.
-
And I'm using the CSV file format.
-
You'll notice there's also a TXT file format, that's probably
-
tab delimited so that the sort of data records
-
have tabs in between them instead of commas.
-
Again, there's a variety of different formats,
-
but the CSV is the one that I'm going to use.
-
Time to start coding.
-
So let's check in and see if you--
-
if you want to follow along, let's check in
-
and see if you have exactly what I have.
-
So what I have is some boiler plate HTML.
-
It's just a plain index of HTML file
-
with a title, a head, a body, and an empty script tag.
-
I've got links to where the data is coming from,
-
just to make sure I'm referencing and crediting
-
properly in my code.
-
And then I also have that CSV file itself.
-
So, yeah, I'm in Visual Studio Code.
-
And you can see, there's my index of HTML
-
and there's my CSV file in the same local directory.
-
But you might be using a different text
-
editor or a different environment, all of this
-
will work as long as you have your HTML file
-
and your CSV file.
-
Let's take a look at that CSV file.
-
So here's the CSV file.
-
You can see that there are a number of columns: Year; Glob,
-
which I assume stands for global; Nhem ,
-
northern hemisphere; et cetera, et cetera.
-
And then you can see that the columns of data each
-
being separated by commas.
-
So this isn't really meant to be human readable.
-
There are ways of viewing a CSV that's more human readable.
-
For example, here's how that same data
-
looks in spreadsheet format.
-
You might notice here, however, that it is colored.
-
Each column has a different color.
-
This is because I'm using a Visual Studio Code
-
extension called Rainbow CSV, which it says if it
-
was like tailor made for me.
-
And I'll include a link in this video's scripture
-
if you want to install that extension as well so you
-
can have things color coded.
-
Another thing I really like to do
-
when I'm working with a data set for the first time,
-
is I like to give myself a test file that
-
has much less stuff in it.
-
Because if I want a console, log, and check stuff,
-
sometimes this big file-- this isn't that big of a file,
-
it's just 1880 to 2018.
-
But, potentially, I could have like a really, really big data
-
file.
-
So something I'm going to do is, I'm
-
going to just quickly do a Save As and call this test.csv.
-
And so now you can see that I have a separate test.csv file.
-
And I'm going to just leave two years in there.
-
So I'm going to scroll all the way down and delete everything.
-
And now I have a CSV file that just has three rows in it:
-
the header row and then the data for 1880 and the data for 1881.
-
So I'm going to work with this first.
-
And once I have the parsing and everything
-
I want to do working properly, then I'll load the real data.
-
So the first step is exactly what you might think,
-
fetch test.csv.
-
Let's write that code.
-
I'm going to write, fetch test.csv.
-
And then remember, fetch returns a promise,
-
and I can handle the resolution of that promise
-
when it is finished loading the data with dot vin
-
and any errors with dot catch.
-
But I prefer to use the await and async syntax,
-
so I'm actually going to put this in a function called,
-
get data.
-
I might think of a different name for that later.
-
And I'm going to say the response equals
-
a weight fetch test.csv.
-
So I'm rating a function-- oh, and this function,
-
I almost forgot, has to have the keyword async because it's
-
an asynchronous function that's making asynchronous calls
-
with the await keyword.
-
So the response equals await fetch dot test CSV.
-
Now you might remember, on the web
-
fetch API there are a variety of kinds of data streams
-
that might come in.
-
There is a blob, there's a JSON, there's an array buffer,
-
there's text.
-
And this is what I want to actually use, raw text.
-
Even though it's tabular data in CSV format,
-
I'm going to do the parsing of it manually in my own code.
-
So I just want to receive it as text.
-
And that means--
-
I'm going to say--
-
I'm just going to say table equal--
-
maybe I'll just call this all data,
-
equals await a rate response dot text.
-
So let's console log that data.
-
And let's call the function, get data here.
-
And then let's go and see this running actually
-
in the browser.
-
And here it is.
-
And you can see, there we go, the data
-
has been logged to the console.
-
Now, ultimately, here there are a variety
-
of JavaScript libraries that will parse a CSV for you.
-
And by I mean, parse, I mean figure out
-
where all the commas are and break up the data
-
and put it into objects and make it usable for you.
-
D3, which is a JavaScript library for data visualization,
-
has a parser in it. p5 js, which is a JavaScript library
-
that I use a lot on this channel,
-
has a load table function, which will actually
-
parse the CSV for you.
-
And there are many others.
-
So I'll include some links to those in the video description.
-
But I think it's a useful exercise right now.
-
It's a simple enough data for us to do the parsing manually
-
with the split function.
-
What?
-
What split function?
-
What are you talking about?
-
So the JavaScript string class, any time
-
you have a piece of text in a variable in JavaScript
-
it's a string object, has a function called split.
-
And that function allows you to take any arbitrary text
-
and split it up into different elements of an array.
-
And that's basically what we want to do.
-
We want to split up all the rows, and then each row,
-
we want to split up all the columns.
-
The split function requires a single argument, a separator
-
or otherwise known as a delimiter.
-
And in this case, we have two kinds of delimiters.
-
For each row, the delimiter, the thing
-
that differentiates one row from another, is a line break.
-
So first let's call split with line break.
-
Going to my code, I can say, and I'm going to call these rows,
-
the rows equals data dot split.
-
And I'm going to split by backslash n.
-
So backslash n is an escape character sequence
-
that indicates a line break or new line.
-
Depending on your file format, you
-
might need a slash r, also which is like a carriage return.
-
You can also use something called a regular expression
-
here.
-
This should also work.
-
Instead of in quotes, if I have forward slashes,
-
the delimiter is a regular expression.
-
What's a regular expression.
-
So that's beyond the scope of what
-
we're doing in this particular video,
-
but I think expressions are so useful when doing string
-
parsing, that I will also link in this video description
-
to a whole series of videos that I have about that.
-
But for now, just the backslash n in single quotes
-
should do for us.
-
So I'm going to say console.log rows,
-
just to make sure that works.
-
And it does.
-
So we can see here, this is the raw text.
-
And now this is split into an array with three elements,
-
each element is one line in that array.
-
And one thing, though, we don't actually need to first row.