That will be the answer for all of these problems, and I admire them for getting stuck in and getting involved.

But one thing that always strikes me is people say, Just change this bit of code, just change this value.

And it strikes me that if we invent a JIA your happen upon the Magi, I The reality is it's probably gonna be in your own that way.

You don't actually know exactly how it's working anyway.

What, you pass on that?

Yeah.

I mean, uh, to yeah, to the people who think they've solved it.

Either You're smarter than everyone else who's thought about this problem so far by a big margin, or you're missing something.

And maybe you should read some more of what other people have thought and learn more about the subject because it's cool.

Like I think it's great that people come in and try and solve it and try and find solutions, and we need that.

But the problem is not a lack of ideas.

It's a it's a lack of good ideas.

This kind of coming in from the outside and saying, Oh, yeah, I've got it I figured out the solution is obviously arrogance, right?

But the whole artificial general intelligence thing is sort of an inherently arrogant thing.

It's it's quite hubristic, you know?

I mean, you talk about playing God making God like, but also that that new I've got it.

This is how you do it.

Sometimes that doesn't work that approach.

Yes, sometimes because you see stuff that people have been too close to the metal, too.

We're close to the problem, Ray, right?

Yet it's Sometimes you get too close to it.

You can't see the picture, you're inside the frame.

Whatever it's, it's totally possible that some random person is gonna come up with a riel workable solution.

And I would love I would love that to happen.

I think that would be the best, because then everyone would have to try and figure out how to cite a YouTube comment in a research paper.

I presume that is already style advice for that anyway.

But the problem is from the outside view, you get a 1,000,000 of these right, and so you know, a 1,000,000 minus one are gonna be not worth your time to read, Which means even the good one isn't actually worth your time to read on balance Because you want you How you gonna differentiate it?

So the only thing you can do is get up to date on the research, read the papers, read when everybody else is doing made sure that what you're doing is unique and then actually write something down and send it to a researcher.

You write it down properly and make it clear immediately up front that you're familiar with the existing work in the field.

Um and then, you know, then maybe you're in with a chance.

But what will probably happen is in the process of all about reading, you realize your mistake.

It's still worth doing.

You've learned something.

This is part of why I safety is such a hard problem.

I think in the sense that a problem can be quite hard and you look at it and you can tell it's quite hard.

A problem that's really hard is the problem.

You look at it and then immediately think you've got a solution and you don't because then you don't even You're like the It's like you're like the satin half, right?

But you're confident Lee with a wrong answer now, rather than at least being honestly uncertain.

Yeah, like ledge ability in machine learning systems is really low right now.

Get kind of black boxes, right?

They're not.

They're not eligible and that you can't easily tell what any given part of it does or how it works.

And that is a real problem for safety.

Definitely.

I think right now, right now, the stage were out with a guy.

Safety is, we're trying to specify any kind of safe agent, which is, you know, trying to build something from the ground up that will be safe.

And I think that's much easier than taking some existing thing that works but isn't safe and trying to make it safe.

I don't think that approach, to be honest, is likely to be fruitful.

I give a really dodgy example of how this might kind of being something people get the grips with, which is the Star Wars scene where the robots are given restraining bolts.

R two d two says, I can't do that unless you take this restraining bolt off, then probably runs away.

So I guess you're too small to run away on me if I take this off.

This is kind of like retrofitting some kind of Australian bolts.

Yeah, I mean, so there's different things, right?

Building an unsafe aye Aye.

And then trying to control it against its will is idiotic.

I think having some of those controls of ways of keeping the system you know, limiting what the system can do and stuff is sensible.

But it's so much better to make a system that doesn't want to do bad things than to try and keep one in.

Um, so this is kind of like the idea of coming to some books, right?

Yet it was like I mean, constraining.

I necessarily means Outwitting it.

And so constraining a super intelligence means Outwitting a super intelligence, which kind of just by definition, is not a winning strategy.

You can't rely on out watching a super intelligence.

Also, it only has to get out once.

That's the other thing.

If you have a super intelligence and you've sort of put it in a box so it can't do anything that's cool.

Maybe we could even build a box that could successfully contain it.

But now what?

We may as well just have a box, right?

There's no benefit to having a super intelligence in a box.

If you can't use it for anything, it needs to be able to do things.

And I properly, properly contained may as well just be a rock, right?

It doesn't do anything if you have your A I you wanted to do something meaningful.

So now you have a problem off you got something you don't know is benevolent.

You don't know that what it wants is what you want and you then need to.

You presumably have some sort of gatekeeper.

Huit, tryst.

It says I'd like to do this and you have to decide.

Is that something we wanted to be doing?

How the hell are we supposed to know?

I mean, how can we if we outsmarted?

How come we reliably differentiate actions?

We want to allow it to take from actions we don't.

And maybe the thing has a long term plan of doing a bunch of things that we don't notice at the time or a problem Until now.

Then can get out right Actually, this speaks.

This speaks to a more general thing, which is there's often a trade off between safety and effectiveness, like with anything, right?

Anything you're designing, there's gonna be You're gonna be trading off different things against one another.

And often you can trade in some effectiveness to get some safety or vice versa.

So some of the things in this paper like that whether thing does become less powerful, then, uh, may I designed differently, but it also becomes safer.

You know, that's always the way it is.

But it's just the way you put your resources, I suppose.

Isn't it right?

But it's it's kind of inherent to the to the thing.

Like I mean, this is true of any tool, right?

The more powerful the tool is, the more dangerous it is.

And if you want to make a powerful tool less dangerous, one of the ways to do that is gonna involve make me less powerful or less flexible or less versatile, or see you know something that's going to reduce the overall effectiveness of it as a tool in exchange for more safety.

And it's the same with a I and obviously gonna be a server for whatever product you're using.

Now.

Any time that Bob sends out a message is going to go via the server by definition.

Because that's the thing that relays messages to Alice.

It knows how to communicate with Alice.

You know who knows what her phone number is?

It has a list of your contacts and things.

You know, this is how it works.

This could be a phone provider.