Okay, so my talk is titled the browser as a modular, networked video synthesizer.

Um, and everything I'm going to talk about is online in the form of sort of a code editor and graphic surrender at this at this web.

Say, um hydra editor dot glitch dot me.

Um, and this is sort of a tool I've been building.

But really, it's more of an investigation of three different things that might seem kind of separate collaboration in the creation of life visuals, possibilities of peer to peer technology on the Web and feedback loops, um, and has also become a platform for live coating visuals in JavaScript.

Um, so I'm gonna start with a reference from the 19 seventies.

This is the satellite arts projects by kick Galloway and Shay Rabinowitz.

And what they did was using using satellites in 1977 they streamed video from two different places and composited them on top of each other.

And what they said was that this marks the first time that the geographically dispersed Elektronik image was contextualized as a live immersive place where artists and others could convene and co create together on a scale that could be a CZ cultural inclusive of desire.

And so this is before the Internet existed.

This is just satellites, and they, uh, applied to get access to this this satellite and to send video from one place to another.

Um, and it's sort of this utopian idea that as soon as maybe if we just put two images on top of each other, we create this third place.

It's not here, it's not there.

But it's this other place that some hybrid of these thes two places, Um and I want to stay here for a second because I think, um, we still use this idea of a place in talking a lot about the Web.

It's a website as if you're going somewhere or it's an address.

Um, a Web page.

Just this idea that when you access the website, your you're going to a place.

Um, and it's sort of this analogy that doesn't necessarily have that much to do with the underlying technology's like, if I go to a website, my eyebrows are sending a message to a server somewhere.

That server sending messages back to me, I'm sending messages back, but it's not.

There's not that there's actually a place that I necessarily, um And then also this idea that it's live an immersive.

I think Andres is talk.

It was a good example of maybe we're just getting right now to this idea of a live immersive place on the web.

Um, so I wanted jumped a little too now, which is, um, and talk about Weber T C, which is a browser protocol for streaming video, audio and data directly between clients.

So this means that now in the browser, we can be a little less dependent on a centralized server.

Um, the server here is this pink thing or some some way that all of these clients can talk to each other.

But then Weber TC says that I can, as a client with a brother, directly send video, audio and data, um, to another person who's connected.

And in this diagram, I'm I'm sure a lot of people have seen it before, but but says maybe with this cut these kinds of decentralized protocols, we Congar go away from the left hand image which is more essential eyes network and go towards the right hand image, which is more of a decentralized network were where lots of different people can connect and exchange information in different ways.

And there's a lot of exciting projects.

I think being built on these this type of protocol iCal, for example, Webb Thorne, speaker browser, the dot protocol, um, and stuff like that.

And I just kind of got interested in this protocol and started doing experiments sending video, balancing video around the Internet.

Um, and as I was doing that, I sort of started to feel like this.

This thing on the rate of, uh, this network looked a little something like this thing, which is another reference from the seventies because percent reason, a lot of my references are from the seventies, huh?

It's kill him from San Francisco.

Uh, but anyway, so what this is is an analog video synthesizer.

Um, and and basically it's a bunch of boxes.

Some boxes create a signal by the time that signal gets, so the screen on the right, the pattern of the signal creates a pattern, and the boxes are designed in such a way that many of the boxes can receive a signal and transform it.

And then So the order in which you connect the boxes will affect what comes out on the the final screen.

And this is much similar in analog modular synthesis for audio as well.

A CZ analog modular synthesis for video.

And so I thought, What if we kind of treat this thing?

This thing kind of like like this thing instead of thinking of of a website as actually a place.

What if we think about it as a bouncing signal that we can play around with?

Um And so So I started experimenting with this idea about 2.5 years ago.

Something, um and so what?

This is, um, is basically here.

I'm using Weber to see to send a video signal between various browser windows.

So here I'm sending this in this video to this window and compositing them with each other.

And then I'm sending this video in this video here and compositing them with each other over here.

Um, and these air all happened to be on the same computer to kind of show the idea, but they could be, you know, when one window could be in China and one some Well, one here.

Um, but this is I don't know why this was interesting to me.

It was just so let me explore this idea and because I didn't know what I wanted to do with it instead of actually making a user interface as I was experimenting with it, I just live coated in the consul kind of playing around with different things and and seeing what happened just with this sort of routing video streams and throughout the Internet.

Um okay, um and so Daniel Sand in the creator of that, uh, synthesizer that I was just showing key.

He said, Ah, the modules are designed to maximize the possibility of interconnection, thereby maximizing the number of possible modifications of the image.

And so the idea, his idea is that the more of those boxes that you have in the modular synthesizer, the more you can route signals may be, the more possibilities you have.

Uh, and so now I'm gonna move into the tool that I've been building to experiment with this this idea.

It's called Hydra, and it's at that link I showed earlier called hydra editor dot Glitch taught me, um and so it's designed to let people experiment with with what I was just showing you not just within by sending video, two different windows, but also just all within the same video with within the same window.

I mean, um and so I'm actually going to get rid of what's on the screen.

Um, and so the basic, um, unit within this is isn't oscillator.

Um Oops.

Okay, so here we have an oscillator which eyes like one of those boxes in the modular synths world.

And then, um, it's in order to see it.

I need to connect it to a screen.

So here's ah connected the US later to an output.

Um, and you can think of in this case, this is a job script library that I've created for working with this model of sort of video synthesis, and you can think of each function as if it's one of those boxes that we're seeing in the modular synthesis world.

So now that haven't oscillator.

I can give it certain parameters like frequency, um, sink and also separate the colors somewhat.

Um, and then I can also start thio transform the signal so they can make it rotate.

Hit.

I can make it rotate more.

Uh, I can make it into a kaleidoscope, for example.

Um, I can, um, pixelated it, um And so so here.

Sort of.

The analogies are each.

Each function is one of the boxes in the modular synthesizer.

Each parameter is kind of like a knob.

And then the chaining of the functions acts like the patching cables people more from who are familiar with modular synthesizers.

Um, and this is this is interesting to me.

This is all running on Web D l and this chain of functions renders to a single shader.

And I was interested.

Um, I think a lot of abstractions for dealing with graphics either talk about canvas like, which use a lot of drawing metaphors.

So there's a canvas.

You draw shape, you draw another shape, Um, or that you sort of three d metaphors.

Like a lot of what we just saw, which is, um, a box lighting a scene.

Um and so here I am using this other metaphor, which is the the video synthesis and transforming a signal metaphor.

And so with this, if you want to repeat something instead of writing a loop and saying, I'm going to do it now 100 times it just apply a transformation.

So here, I could do Repeat, and so they could do it two times.

Uh, or I could do it 200 times.

Um and it's not more proud.

It's not more intensive for the process for the processor at all, because they're just transformations to the coordinates directly on the graphics card.

And the result outputs a pixel color.

Um, and so that's all well and good, but this is just sort of a single chain of things.

And so, uh, what?

I was really interested in more that mess of cables and how things were out into each other.

And so what I can do here is, um, I can have various various sort of output screens, and so, uh here and I can also use different sources.

And so I'm going to use my web cam here.

Um uh, so they actually want to put this over there and that over here and said I have, um I have my to my two screens, and I can then also blend them together.

So, um, everyone up put called 00 and one called Owen.

So then I can now say I'm gonna use Ozio.

I'm gonna blend it with a one.

And then Evan output it Thio two.

And so then, uh, there, Um and there's something different sort of blending operations here, but I'll make this a little, um, and so one other thing that I sort of, um, took for modular synthesizer these air.

What happened?

What's happening?

That was I'm combining two colors and the two colors of the two images on the left create the colors on the right.

But I could also use the colors of one thing to affect the coordinates of the other thing.

And so that in the analog world is called modulation.

And so I can, um, do that.

So now you it's the colors of the thing on the top are affecting the coordinates of my camera.

So it ends up with this kind of worth anything.

Uh huh.

And so, um hi, everyone.

I'm gonna repeat myself, And so this is just something I I made this to experiment with and, um, uh, a lot of and also back to the earlier image, showing all the windows.

Still, if anyone else is connected to this website, I can use what they're doing as an input to what I'm doing, which is sort of just a experiment of of what happens when we come by.

An image is in this way on.

And then some of the other things I have experimented with this.

This is like using the screen as an input to itself in creating feedback and this sort of trippy thing.

Um, well, uh, here I'm using my Twitter feed, and and, uh, also sources in the software can either be someone else who's connected.

They can be a camera.

It could be any browser window.

Um, or it could be any application window, or it can be a canvas element.

Um, and so here here, this is my Twitter feed with Cem video effects of play to it.

Ah, or this a use, um, a live stream of times Square in a live stream of an aquarium and then live coded the mixed between the two.

And this.

For me, it's really just an ongoing investigation of trying something out and seeing what happens and try something else out.

Um, in another experiment, this was using, um uh, chromium with no background and so I could actually just use my desktop as input to this video synthesizer.

Um, but with with her without having windows.

So that was just a fun experiment.

Um um And so one of the interesting things to me with this sort of set up is to be able to make it easy to experiment with feedback, which, um, in in the video world.

Feedback is where you have, um, a camera and output over the camera goes to a screen, and then the camera is looking at that screen itself.

And so, uh, there's no at some point, there's no starting point because the image of the camera is on the screen.

And the image of the screen is what's coming out of the camera, which is similar to Rikers, Asian and programming or some of ushers, ushers, drawings, or, in the 10th century, the snake that eats its own tail.

So it's this idea of things that don't necessarily have a start.

Um, so I'm gonna show a very quick example of that here.

It's also can be useful for making fractals.

And so ah, here, I'm just gonna output a shape.

Can you see my code, or should I make it bigger.

It's good.

Okay, so here you have a shape.

Um, and then I'm gonna blend the output of that with, uh, with itself.

So the output here is called Ozier.

Trust me on that.

I guess I'm gonna blend the output with it, so and we don't actually see any difference because we're drawing a triangle, and then we're mixing it with itself again, which is a triangle.

Um, but I could after each time I makes the whole thing repeat, repeat it and then mix it with itself again.

And so to do that with this, um, I could just see a repeat, and we get a sort of fractal like thing, which is, um, something that at different scales is a repetition of itself.

And and it's something that I'm just playing around with.

That's really interesting with this kind of set of abstraction is to be able to in only four functions.

Kind of describe.

Describe a fractal.

Um, I'm gonna do one more little little demo.

Um, it's Phoebe.

I'm gonna start with the camera again.

Hi.

That's my face.

Very big.

And then I'm gonna blend it with the output, and then I'm going to use that, um, modulate function that I just showed you where the colors of one thing affect the coordinates of another thing.

So here, Marge.

So I get this sort of liquidy thing.

Um, they could add a little more contrast, maybe.

Let's see, maybe a little more.

And so it turns out that that feedback is useful for modeling.

Muh, muh, muh muh modeling things like fluid dynamics as well, because, um you can think of of the colors of the texture is sort of a representation of pressure and then on each each loop sort of affect the system using using colors.

And, um so, yeah, some things I've made, like experimenting with this just this pretty simple, simple setup.

Also, it can be useful for creating reaction diffusion like like patterns.

And so, uh, one thing that the creator of that module, a synthesizer that I showed at the beginning, talked about, um, he talked about the idea of this feedback loop which has had sometimes like, a mystical, uh, mystical implications at different moments in history.

But he says, the reason you put yourself in the feedback loop is so you can learn something I'd rather learn to do something, you know, you do something and the results get processed somehow, perhaps just by yourself, just looking at it.

And you do another thing and the results of that get processed and your actions can cause constantly be tuned or corrected or find by what you see and something that happens with playing with the systems that can be kind of delicate is that even though I know exactly what each function does at the moment of fling with it, a lot of times a really unexpected thing can happen.

Like like in this example.

If I make one of these variables, just change one number, maybe it makes it, really It goes away now.

And so, uh, it can be really delicate.

And so, being ableto live, code it or toe, I'm actually executing parts of the code at a time and being able to code it, see the results code it see the results to me is very similar to putting yourself in that feedback loop with which, in Anna, analog world is like patching cables together.