Week 3 Take Off that Silly Watch Go back ◀
Wearables come in many different forms. They can be embedded in shoes, clothes, watches, and more. But what about sensors that you don’t have to wear? What would it look like to make a step counter that doesn't require a smartwatch or other wearable?
In these activities we are going to look at some technologies that can be used to help people practice sports that primarily use a camera and/or a microphone. Many of these technologies are very much on the cutting edge. To give you a sense of this history, take a look at this video from November 2006 when Nintendo first came out with Wii Sports. Wii Sports was arguably the first video game to have people wirelessly move around to control the gameplay experience.
Note: There were some games that involved movement back in the 1980s. Check out this video on the Nintendo Power Pad if you are curious -
The Nintendo Wii still required users to hold a controller. All of this changed with the XBox Kinect. Here is a short video about playing sports video games without a controller from 2010.
Fast forward to the present day and people can practice sports without using any wearables or physical mats. With certain apps, they can even get automated coaching.
To give you an example of this, today you'll be checking out homecourt.ai, an app that helps with skill development in basketball and soccer. Applications like this make it possible for people to practice sports amidst the Covid-19 pandemic. Before we start, here is a short video clip about homecourt.ai.
Alright, now it’s time for you to test out homecourt.ai. First, grab your iPad or iPhone. Next, go into the App Store or Self-service and search for “homecourt.” Install the app. You may need to ask an adult for permission.
Once you have the app installed, you will need to make a user account.
To setup an account follow these steps:
- Open the app.
- Select "Add Player"
- Select "Create a new Profile"
- Choose a First name and Last name (they don't have to be your real name).
- Press 'Create'
- Add a Profile Photo if you'd like. Otherwise press 'Skip'.
- On the next screen it will ask you to select Profile and Workout Privacy. I recommend selecting “Followers”.
- Press "Back".
- Press 'Save'
- Now, to join the Divas Data in Motion Team, click on this link from your iPad.
Now that the app is set up, let’s test out a few activities. Most of the activities take about 1-3 minutes to complete. You’ll want to give yourself a little bit of room to move left and right. If you have access to a basketball, there are many activities that use one. The activities that are suggested below don’t require a ball.
The first activity is a jog-in-place step counter drill. If you have it available, grab your Micro:bit step counter or Moov watch to compare with.
Compare and take notes in your Data in Motion Journal:
- How many steps did the homecourt app record?
- How about the Microbit and the Moov watch?
- Why do you think these would be different, if they were?
Next is the Reaction drill. Try it a couple of times and see if you can beat your previous high score. Maybe even challenge a family member to a battle.
Lateral Quickness drill, is the last one to try before going through the app and picking out one or two activities/drills that look interesting to you.
Now that you have tested the app, imagine that you are a professional reviewer who is rating how good the app is.
Answer in your Data in Motion Journal:
- Write down ideas about how the app works.
- What worked well in the app? What things didn't work well?
- Who does this app work for, and who wouldn’t this work for? What kind of assumptions are made about the user of this app?
Click 'Launch Episode' to join the session. Or follow this link: Zoom Meeting Link
During today's sessions you will be making a reaction time game using Scratch. A working copy of the program can be found in the resources section at the bottom of this page. The next several paragraphs include step by step instructions on how we wrote the program.
--- Skip to the end of the page to get the Scratch Code ---
The same tools that are used to run the homecourt.ai app are available to anyone in the world using Scratch. The term that people use for this technology is computer vision. Basically, computer vision is about helping computers recognize and see things similar to the ways people do. During today's activity you will build your own version of the reaction game. As a refresher let's take two minutes and test out the Reaction game one more time.
The basic elements of the Reaction game are:
- The game background has to be properly setup before the user can start
- The game needs to provide instructions for how to play
- The user needs to actually play the game with targets that appear to random places.
- The game gives the user information about how they did.
We are going to walk through making this game together. At the end, we are going to ask each of you to customize the game.
To get started, everyone should to go https://scratch.mit.edu/
You will start with a project that has a lot of the parts filled in for you. Here is the link for anyone that doesn’t have it already: https://drive.google.com/file/d/1DPvpW8IFXiJ15ULQR6K6PKIlsRUe0_Qs/view?usp=sharing
Make sure to download this code to somewhere that you can remember.
Once you are back at the Scratch website, select ‘Create’. A new project will load. In the upper corner, select File-> Load from your computer.
Then select the Scratch file that we shared with you. Now that we all have it opened
I’ll quickly explain what each of the different parts does, and then we’ll work on finishing up the game.
The code is broken up into five chunks.
The first one has the green flag trigger. Some of you will probably remember this, but programs in Scratch start when you press the green flag. Underneath the block with the green flag are four chunks of code that correspond to the four game elements that we mentioned earlier. First we setup the game, then we show the instructions. Next we run the game. And, finally, we show the results and end the game.
Let’s look at the setup the game part. Here we have four blocks. First we hide the basketball, make it go to the center, set the background transparency, and then turn on our video.
The next major step is to give the user instructions for how to play the game.
This chunk of code will display instructions for the user, and give a five second countdown.
You can feel free to change the text of the instructions by clicking on the text in either of the purple blocks.
Now we are on to the part which actually runs the game. This is where you all will add in some code. To run the game, we need to make the basketball appear at different places around the screen, and then make it disappear after a set amount of time, or we swipe near the basketball. For this part we will be using another loop. But before we start the loop, we are going to make a new variable that will keep track of how long it takes us to swipe the different basketballs. Scratch has a built in timer, but we will need that for something else, so, we are going to make a new variable called “total timer” and set it to zero as the next block in our program.
Now we can add in a “repeat …” loop (“Control”). For the repeat loop, we can decide how many times it should run. This will determine how many times the user needs to hit a basketball. I am going to use 10, but you are free to make your game longer or shorter as you see fit.
The basic steps that will take place inside the loop are as follows.
- Move the ball to a random position.
- Make the ball appear.
- Reset the timer
- Wait until motion is detecting near the ball, or 8 seconds pass on the timer.
- Add the time to the total game time timer.
- Hide the basketball.
The steps above directly correspond to the blocks that we will use. First we will use the “go to random position” block in the Motion blocks.
The next block we will add is the “show” block under Looks.
This will be followed by a reset timer block which can be found under Sensing.
Now we are on step 4 above. We are going to use a “wait until” block(“Control”). Inside that “wait until”, we will add an or operator (the green blocks).
And inside the “or” operator, we will put in two “greater than” operators. The first “greater than” operator can be for the timer. So the first oval will have the ‘timer’ oval from Sensing and the second slot in that “greater than” operator will be 8, to correspond with roughly 8 seconds.
The second “greater than” operator will be for the motion sensing. So, on the larger side of the “greater than” operator, we will drag in the “video motion on sprite” block(“Video Sensing”). In the second oval of that or operator, we will put the number 40.
To finish up this part we need to drag in a “change … by …” block, and select total timer from the drop down menu. In the second part of that block you will drag in the timer variable. (Remember this is keeping track of how much time someone takes to touch the targets).
Lastly, we will add a “hide” block from the Looks blocks.
Go ahead and run your program, and let’s see how it is working.
The last chunk of code is for showing the results and ending the game. It turns off the camera and let’s the player know how fast they were able to hit all of the targets.
Go ahead and play the game some..
Homecourt.ai has a few other features that we didn’t include in our game, but that could reasonably be added. For example, some of the homecourt activities track your dribbling speed. One way to do this is by listening for a loud bouncing noise. You can do this in Scratch by using the loudness or volume variable. Homecourt also uses body pose estimation and people detection. We explore some of these approaches in another activity.
This is a lot of cover in one session. Thanks for hanging in there.
One thing for us to reflect on is who gets to make these applications and why. When we watched the videos that talked about designing video games, many of the designers were white men. How do you think these applications and games would be improved if they were designed by girls and women like us? In a later session we will talk more about how limitations in the people who are designing these technologies introduces significant problems in how those technologies are used.
For now, keep working on the reaction time activity or try creating something new. You can add in sounds, or have a family member try the game.
Be sure to share what you make in your Data in Motions journal.
The Data in Motion team has identified several inspiring projects that use wearables with sensors in sports. Many are led by women, and highlight the unique perspectives and important problems that women have been addressing. And we are not just talking about contributions to the sports that people frequently watch on television like football, basketball, softball, and soccer. Women are also innovating neat ways to use these technologies in areas like ballet, hip-hop dance, figure skating and more. Take a look at these videos, and start to think about any ideas that you might have for using wearables or sensors in the physical activities that you enjoy.
This first video is from a project called E-Traces. This ballet dancer instrumented her ballet shoes to create traces of her movements.
E-TRACES, memories of dance from Lesia Trubat on Vimeo.
The next video is a tool called DanceOn. DanceOn in a programming notation that was specifically designed for dancers to to augment their dance videos using computer vision and neat graphical overlays.
If you want to test out DanceOn you can do so at - https://dancinganddabbing.github.io/danceON/
This third video describes STEM for Dance in New York City. Many of the experiences are similar to activities that you all do in Divas, but with a central focus on dance.
The next video is a project out of Brigham Young University where researchers wanted to study the biomechanics of figure skating and think about the amount of strain that it puts on the body.
This last video clip is from an interview with Ivonna Dumanyan, the engineer behind Fathom AI (fathomai.com) - Fathom Ai includes a wearable and a smartphone app that helps athletes avoid injuries. In the video she talks a little bit about the application and what it’s like to start a company.
Take a moment to write down some ideas of wearables that you think would be useful Put these ideas in your Data in Motion journal.