Hey, everyone there is for those.

You have never telling these up here.

It says on air just randomly.

It's like you count Stands like on a welcome back.

I'm Chris Williams with Matt Hamasaki.

We're doing robots Weekly.

This is episode number four mil.

Let Matt introduce our guest.

We're going a little bit off of the train.

From where we've been, We've been mainly no jazz focus.

But that's by no means what we want.

Robots weekly to be.

We're going to go to all different languages and operating systems and also to fund stuff anything that a little you know, small developer heart could want to hear about with respect of robots.

This is the place.

And this is where we're gonna go with it.

So whoever's had big hearts, you know, different developers than ideo Big hearts and infinite patience.

I say great.

All right, so today we have with us Ah, Ben Lower.

He has a, uh, developer, uh, community manager at Microsoft, especially on the eye of the connect for Windows team.

Ah, So one of the things that chris and I were asking is is what are people really doing today with with really cool electronics and the connectors is one of those things that kind of sticks out.

And, uh, especially with the release of of Connect for Windows.

Uh, the hacker community is kind of embraced in weird and interesting ways from anywhere from controlling.

Ah ah, controlling new user interfaces to controlling robots with it.

So with that, Ben, welcome to the show.

Great.

Thanks for having me, guys.

All right, so So you're a community developer manager for ah ah, for Microsoft.

What does that entail exactly?

Well, essentially, the way I think of my job is my job is to help developers out there in the world be successful using the connector windows platform.

Um, so that entails developer support.

It entails, you know, speaking, doing events.

Uh, helping Thio get the word out about what's new.

What?

We're working on new releases.

So it's really just kind of working with all the developers out there.

Who who, uh, who want to use connect for windows is part of what they were working on.

Okay, great.

That's awesome.

So when we talk about connect for windows, how is that really different from connect for Xbox.

And what was the reason behind that?

Well, I'm assuming that that Ah, most of your viewers are gonna be familiar with the connect the product that Microsoft shipped a couple years ago.

I started off as connect for Xbox 360.

And so really, what we did is you kind of mentioned the hacker community already.

They've embraced it in different ways.

And, you know, as a company, we always saw that there was a future for connect beyond game beyond the living room.

And I'll just say that I think that the enthusiasm that we saw from the hackers out there around that, you know, bringing, you know, doing the Tom Hanks big piano and all those kind of crazy things that we saw right away.

Um, you know, I think that really just kind of sped up our timeline for broadening the support of the Connect platform beyond the Xbox.

So connect for windows.

Think of it as you know, connect for Xbox.

360 is designed to work with the Xbox.

It plugs into the X box.

It's the in the living room gaming scenarios.

We took that same technology.

That's basically the same sensor of the depth sensor, that infrared emitter, the microphone array the color camera, and we package that up into, ah, version that plugs into a computer.

It has the license, a commercial deployment license so that developers, hackers and businesses are able to take it and deployed as as a commercial product on.

And then we also have, ah, sdk and developer tool kit that comes with that that's downloadable free of charge for people to go and start building APS and ah, you are tool kits full of different sample applications.

All the source code is there for people to use.

Um, you know, take learn from tweak whatever they want to.

D'oh, that's what.

That's great.

Yeah.

Could you go over some of the samples that are included?

Um, for example.

Ah, for example, of things like gestures.

How do you get started with those kinds of things?

Yeah, sure.

So the this I would say the it for people who are new to connect development.

The sample that I always start with is called are connected floor sample, and I like to think of that is sort of your dashboard on connect.

So when you run that application you actually see all of the data streams that are coming off of the connect.

But you see them represented in a way that makes sense.

So, for example, in the bottom left corner, you see this sort of dial?

Um, that shows you the audio beam forming.

I don't know if you guys know what that means.

Is that for anybody who doesn't who's watching?

It's a fancy way of saying we know from which direction a sound is coming, right?

But you see, then that microphone array at work there in the U I for connect Explorer, you can see yourself with the 20 point skeleton mapped on your body.

You cantata ll that on and off you cantata Galan seated skeleton mode, which just gives you the top 10 joints of your body.

So connect Explorer is really a great tool for people to get their head around.

What are these data streams coming off the sensor?

How might I start using them from there?

We have samples that show face tracking.

We actually map a mash of over over 80 distinct points on your face so you can see if somebody raises their eyebrows or if they're smiling, and you can get that data back and do interesting, interesting things with it.

But Matthew us specifically about gestures.

I'm trying to think of, ah, particular sample where we do gesture.

You know, the thing is, his gestures are really, really hard to get right arm because, you know, what we've learned is, you know, people think, Oh, if I just do a swipe gesture, you know, I just do this.

That should be easy, but it really depends on how you've implemented it.

You know, I might actually gesture with my elbow out and you might gesture with your elbow.

And But we actually have some developer content.

Some hands on labs that we've published online would show you kind of like going from zero to a basic kind of photo viewing app that implement some basic gestures.

And there's there's been a ton of work out in our community who people post things on code plex and elsewhere, showing things like, Yeah, absolutely one of the more interesting samples, at least in my opinion, that's that's come out in the in the past release or so it is the idea around Connect fusion, which is Theo idea that you can scan a room, take a take a connect unit and literally just walk around the room and kind of scan it.

And could you tell us a little bit about about connect fusion?

Yeah, sure.

So Connect Fusion is a technology that came out of Microsoft Research.

Ah, oui, my team.

As do many teams that Microsoft worked very closely with Microsoft Research on and we were essentially ableto product highs.

The work that they had done so connect fusion as you're describing, Matthew, Um, I think of it as it's a three D scanner is another way to think of it.

But basically what it does is use the connect and it generates.

Ah, riel.

Time three d mesh or point cloud of an object.

So, um ah, I actually have Ah, I've scanned myself.

I've actually had someone else came in.

It's kind of hard to scan yourself sometimes, but I've had somebody else scanned me and then out Put that to a three D printer and ah, 11 fun thing that I did is I don't know if you remember, but when Nokia shipped to the Lumia a 20 they actually released the actual three D printable cover for that fun.

That's right, Yes, I don't know.

And it's up on thing verse.

So we took that and I said, I said, You know what?

No Queue came up with their first phone.

They said it was made of polycarbonate, and I always said, Well, that's the stuff that Han Solo is frozen in, right?

It's sort of my joke.

So I have myself printed coming out of the back of a limo 20 case that we were able to do because I did the Connect fusion scan and then, you know, sent that to a three D printer.

So, um, you know, for people who are into three D printing connect fusion could be an awesome technology for model acquisition.

You know, getting that three D point cloud data into your cat or three D three D software.

Or, like I said, sending it out to a three D printer.

But, um, you can scan small environments, your desk or whatever it might be.

Have you guys tried that?

I know that maker but just released their digitize Err on and I know that 81 Windows 81 has a lot of the three d printer, especially for maker Bob Built in.

Um, have you guys tried taking objects from your world, digitize it and then generated out in plastic?

Uh, yeah, we've done that.

We've done that dozens or hundreds of times.

We've got it.

We've got to make her about here in the office and we do that kind of thing all the time.

How does Thea how does the transport from one to the next to the next convey in terms of quality?

Because I know that with the digitize er make about digitizing, they had a whole bunch of problems with getting the skeletal structure down and the model rap for it, especially on the precision that you need to make it a quality duplication.

Um, this I would assume, with the way the connect is at least built with the different availability of the connect that it would be able to do, it better would be a way out for is it or at least differently?

Yeah.

I mean, we obviously are doing different technology mean my understanding of the maker about digitize er is it's designed for kind of ah, like an eight inch object or below right?

I think it's designed to sort of be like a small Replicator.

As you kind of said, Um, where is the connect with Connect fusion?

You can scan an object that's eight inches to Aiken, scan my entire body and then have that as, ah, as a three D like an O B J or nest yell file and then clean that up.

I think the challenge that we see isn't so much around quality, if you will, in terms of like getting the pixels, it's Maura around, Uh, you know what?

What happens when there are missed areas in the scan?

So, for example, when I was posing for my Han solo and carbonite thing, Um, even though we were really focus on getting a good scan, their the backs of my hands maybe weren't scanned perfectly or the back of my ear.

And so then what happens when you send that out to a three D printer?

Like, how do you fill that in?

So I think that, you know, when you use connect fusion to do a model acquisition, you do have to have some you know you have to spend some time doing model cleanup and kind of filling in holes to make it.

You know what they call water tight.

And then I think it works really well.

And I've got actually, I've got a had something here on my desk, which is like a mini replica of my head here.

I don't know if you could see that here.

That's awesome.

But we were at a maker faire here in Seattle a couple months ago.

We were actually just scanning people that would come to the booth and then doing little three D prints.

So, um and that's a nice thing, too, is even though you're you're acquiring a model who, like myself, maybe five foot nine inches tall, you're able to then scale it up or down.

Um, for for the print, right.

So I think it's a ton of fun, and it's one of those things.

Um, just just to kind of clothes out on the topic effusion.

Unless you have more questions, Matthew.

But in our next release of RST K, which is happening in September, the 18 sdk, we're actually going to add full color support to connect fusion so violent if you If you do a scan with fusion, you kind of get what I call a sort of the gray lump of clay like Yep, I can tell it's me.

It's my full facial structure and everything, but it's just all gray.

And so what we're doing in 18 is reading.

We're adding each of the color pixels, so it's it's mind blowing to see this whole model.

Now, with the full color fidelity added, End it.

It's really cool.

Yeah, especially if you watch the videos from Microsoft Research.

It was it was amazing to watch them, especially when they could when they could track like a teapot.

You put a tea pot down and then you remove the teapot.

It's still remembered where it waas.

I mean that that, to me was was just literally mind blowing that you could come in there to look at a room and see what's changed over time.

Yes.

Yeah, that was That was completely awesome.

So s So you kind of mentioned what was upcoming.

So what?

What else is up coming in the next sdk?

Um, well, some of the things that Aaron 18 and we actually talked about this a bit at the build conference that was in San Francisco back in June.

Uh, you know, we've had a number of developers come to us and say, Hey, we really want to start adding connect to what we d'oh.

But we're Web developers ht mo five Java script.

Those right skills, I don't know w p f or, you know, or maybe they're tooling and their workflow is all around html five JavaScript.

So we're adding that as a supported APP model in 18 Now, we're not building it with the scenario of I have a website and you have a connect at home, and we're kind of doing things over the weather.

It's not like we have our t.

C Uh, no, we're We've built it as a local host Scenarios to think of it as I want to do, um, I want to do like a digital display or a kiosk type of thing.