Body.Scratch gave a demo at TMC Data Science, one of our long-term partners, inviting children to test out our immersive game. For the first hours of the event, multiple families joined in and enjoyed discovering and learning together.
At some point, I saw a little man eating blue candy floss in the background and watching others bodycoding. Impatient to wait his turn, he left. A few moments later, when the room was empty, he came back together with his dad. We will call little man S. He is 9 and he has a lot to tell about technology! I was very please with the interaction, so I am delighted to share with you snapshots of the conversation. S is the boy, D is the dad and A is I.
A: Hello, welcome back! Would you like to play?
Still busy with his candy floss. Looks at the screen then nods his head positively. The session starts, and he really enjoys the warm up with some basic moves. Then 3 minutes in he says,
S: I prefer Scratch better, easier.
Dad smiles at me awkwardly and says to S, D: this is a bit different, keep going.
A: It's all good, it's okay if you prefer Scratch.
Then S proceeds on playing, until after a few minutes in he asks,
S: How does the computer know what I want to do? Because I tried to move this way and it's not exactly what's on screen and it still works.
D: This is very complex...
A: This is a great question! We made a technology that works with the camera, and then we let the computer look at many people dancing and then told it which step matches which block of code. So then the computer compares and checks which move is the closest to what you try to do, and then decides if it's close enough, and validates it.
S: And is it possible for someone without an arm or a leg to play it?
Then he proceeds with putting an arm in his t-shirt to simulate a handicap, while I start explaining, and proceeds with a variety of poses and moves to test the computer. Dad still smiles awkwardly.
D: He really likes technology.
A: That is another great question! Well we haven't focused on that part yet, because it requires to work closely with various people with various conditions and train the computer to recognize their moves.
Then S puts his arm back out and the model recognizes his move. He looks at me and says,
S: Everyone should play this. Also the people without arms or legs. Maybe you can write a script so the computer creates an arm for someone without arm, so you can see it on the screen and it works to recognize the moves.
A: That is a great idea!
S: Maybe you can even do it for someone who misses both arms, so they only have to dance with their legs.
A: Never though of that, that's great you think of them too!
S: Now you can tell your team about it. This is...hoe zeg je dat...toekomst.
A: The future!
S: Ah yes! This is the future! Everyone should play this.
Dad smiles.
S: What is your biggest bug?
A: Well, we ran into a case where, because we didn't have enough people who trained the computer some players like you would not be recognized, because the computer did not know their body proportions.
S: Which language did you use to write it?
A: the part of recognizing the moves, in Python.
D: Ah Python! Next week he gives a presentation to his class about Python and TypeScript.
A: Oh so you are really into coding.
S: Yes. Can you show me your code?
I proceed with opening up an old piece of our Python code.
A: This code is about pose estimation, so the computer maps out the different parts of your body the best it can to be able to recognize the moves the best it can later on.
Then he looks at me and says,
S: Can you delete this part?
D: What?! No, why would he do that?
A: Ahah! I mean I would run into bugs for sure.
S: What would happen? Would it be that the computer just doesn't recognize some moves or would it be complete chaos?
A: Ahah! This I don't know, it could be either depending on which part of the code you delete. It's great that you ask good questions like this.
S: Now you can tell you team about it right?
A: I definitely will!
Seeing the curiosity both of children who are not interested in computer science and those who already are is always refreshing!
