I work in the Intelligent Robotics Group (IRG) at NASA Ames Research Center, and when we got the chance to collaborate with our next-door neighbor Google on their new Project Tango, we knew exactly what to do: we’re sending the Project Tango smartphone to the International Space Station, where it will set our robots free.
Smart SPHERES with a space-ready Project Tango phone. Photo courtesy of NASA.
Our robots on the space station are called Smart SPHERES — they’re based on the Synchronized Position Hold, Engage, Reorient Experimental Satellites (SPHERES) that were launched in 2006. Each of the three original SPHERES is a volleyball-sized free flyer that jets around with CO2 thrusters and talks using 900 MHz radios. The free flyers know where they are because astronauts have set up sonar beacons and programmed the beacon coordinates into the SPHERES ahead of time, letting the SPHERES calculate their positions from the times of flight of the sonar pings. The five sonar beacons mark corners of a six-foot cube, and the astronauts have to stay out of that cube while SPHERES are running — a solid object could block the sonar pings and throw off the position estimate of the SPHERES.
A vacant six-foot cube is perfect for SPHERES’ original purpose, which was validating control algorithms for satellites’ flying information. But we in IRG wanted SPHERES to do more. With sensors attached, SPHERES could do environmental surveys, measuring radiation levels, lighting, and air quality, for instance, so the astronauts wouldn’t have to. Also, if the SPHERES robots were equipped with cameras, they could monitor inventory or give flight controllers in Mission Control better situational awareness. Any of those possibilities would make a difference in the way we use the space station, but any of those possibilities would also require us to upgrade the SPHERES considerably.
IRG upgraded the SPHERES once already. In 2011, we sent two Nexus S smartphones to the space station to give the SPHERES cameras, Wi-Fi, and more processing power. We named the smartphone-SPHERES combo “Smart SPHERES,” and that summer we controlled Smart SPHERES from the Mission Control Center in Houston. We were all excited to see live video from space streamed from our hardware, but we knew it was just the first step. The Smart SPHERES were still confined to their six-foot cube, so our video showed only the inside of the module that had beacons set up in it already.
Enter Project Tango. The Tango smartphone can localize itself in three dimensions using nothing more than cameras, IMUs, a depth sensor, and a quad-core processor — in other words, no sonar beacons are required. With Tango, the Smart SPHERES could leave their six-foot cube and navigate to other space station modules. That expanded range would let our ground-controlled camera see more than one module. Down the road, this same functionality could allow for environmental surveys of the entire space station.
First, though, the Tango smartphone had to undergo a few changes to become space worthy. We swapped its battery for a space-certified battery (PDF), which is tested to work under extreme conditions, including temperature, vibration, and over- and under-current, and then packaged with additional circuitry to prevent overcharging. We also covered the phone’s screen with Teflon tape to prevent slivers of glass from floating around if the screen ever broke. The phone was “butterflied” so that both the touchscreen and the camera face out when it’s attached to a SPHERES robot. We also changed the Tango code so the phone will localize correctly when it doesn’t feel the force of gravity.
To test that our code changes really worked, we needed time in zero gravity. NASA doesn’t have a zero-gravity room, but luckily for us, it does have a zero-gravity plane. Through NASA’s Flight Opportunities Program, we flew on four parabolic flights in February 2014.
Not only was that the most fun I’ve ever had on a business trip, but also our results from those 15-second intervals of weightlessness were good, and now our modified Tango smartphone is scheduled to launch in June.
This summer, astronauts will connect a Tango smartphone to one of the three SPHERES robots aboard the space station and command the resulting Smart SPHERES to fly through several space station modules. The Smart SPHERES robot has no obstacle avoidance, so it will need to know its position precisely to safely steer around corners. IRG will be watching from the ground, rooting for our robot to succeed in its first trip between space station modules.
Even with its newfound freedom, the Smart SPHERES robot has limitations that prevent it from being the robot that Mission Control needs. It moves slowly and requires constant human supervision. But with Tango, at least it won’t get lost in space.
See NASA’s SPHERES demo as well as other demos from Solid on our Solid Demos page.