I woke up in a Soho doorway A policeman knew my name He said you can go sleep at home tonight If you can get up and walk away Well, who are you? (who are you? who, who, who, who?) I really wanna know… —The Who I have two stories to share with you. Bear with me, they're both related. The First Story I was meeting a friend downtown San Jose yesterday. It was a very bright and sunny day; the hard sun was at just the perfect angle in the sky to inflict the most brutal and piercing light through the edges of everyone’s sunglasses, leaving people scurrying between buildings for shadow and using Handmaid’s Tale-style sunbonnets. Despite the clarity of the weather, let’s just say that visibility was poor. And yet, I saw my friend from an entire block away. I couldn’t see his face, I couldn’t see his clothes; all I could see was the edge of his face in sunlight, the shadows connecting his shoulders and arms, and the gait of his walk. And yet, I knew without a doubt that it was he as soon as he came to within rock-throwing distance. (I bet this is an evolutionary trait: determining friend from foe before they get close enough to throw a rock at you.) The Second Story I also have two kittens at home. (No, I’m not going to turn this into a crazy cat lady blog. I promise. Well, maybe one photo. Okay, two.) One of their quirks is that they can’t seem to recognize anyone without going up to them and smelling them, nuzzling into their palms or rubbing up against their ankles. Whenever you come home, they both come running, and then take their sweet time in getting to know you again. If you have been near a dog, another cat, or (and this is weird) a manicure/pedicure place, they will take extra time re-familiarizing themselves with you. Recognition I present these two scenarios to illustrate how incredibly difficult it is to recognize a person. This is not because people all look the same (though maybe the aliens disagree with me), but especially from a distance, our human monkey brains are incredibly adept at taking incomplete information and forming a complete image upon which we can then act. Think of what it takes to recognize someone from close up: you may consider their facial structure, including relative distance between the eyes and nose and the creases of the mouth, the shape of the chin, the positioning of the ears; the skin and hair tones and texture; you may look at eye color; the unique way someone’s face moves while they are talking, the micro expressions that they don’t even know that they're making, not to mention their voice, speech patterns, verbal tics — all of these build up an identity in your brain. Then you can say, oh! That’s John or Eric or Sanjay or Peter! Even from a distance, we can still tell; using an individual’s gait, body structure, body language, gestural expressions—all unique ways that we all express as we move through space. Why can humans make these distinctions from so far away? Other animals must rely on “closer” forms of identification, even if they have good eyesight—scent (pheromones, intestinal smells, products we use on our bodies or clothing, other bodily indicators), voice, and finally body and facial recognition. Think of how your dog recognizes you—you usually have to be pretty close in proximity to be recognized (unless you haven’t showered in a few days). But when you’re waiting for a beloved at the airport? You can recognize them from a mile off. Augmented and Virtual Realities Last week, I attended a presentation at the Cadence Distinguished Speaker Series about AR/VR/IR/MR/OMGBBQR (augmented reality, virtual reality, integrated reality, mixed reality, oh-my-god-barbecue reality) (okay I made up that last one), with Dr. Jon Peddie of Jon Peddie Research. He has recently published a book on the topic, called Augmented Reality: Where We Will All Live . It was a fascinating discussion, exploring what AR/VR is and what it isn’t, what is required to make it work, and what some of the grand challenges will be to making this technology a ubiquitous reality. Dr. Peddie had so many ideas about the possibilities of where this technology is going—some things I hadn’t even begun to consider. One of the technologies that he suggested will be available on the consumer market very soon is an AR app to help identify people: friends, acquaintances, and strangers. You may even be able to pull up more detailed information, such as what you talked about the last time you saw each other. One would expect the accuracy of such an aide to go down the farther away the subject is from the sensor. Still, if we can teach the system to recognize a person by their facial structure, what’s to prevent the system from determining a person’s identity based on their gait or posture or the length of their neck or habit of gesturing a particular way? The amount of computing power to make this kind of system function is staggering. In a single set of non-dorky helmets or glasses or headsets—or even contact lenses—the system alone must be able to accomplish: High-resolution projection with color IR sensing Forward-facing tracking Depth-sensing sensing Directional microphones and audio for projection and recording 6 degrees of freedom (DOF) (plus speed and direction in time, so more like 8DOF) Eye-tracking cameras to sense where the user is looking and blinking Ambient light sensing Bone-connection transducers for sound and power Recording any or all relevant data Enabling connectivity to a device or network Adequate power supply to last a full-length party at the Holiday Inn …Not to mention the compute power to make the " machinelearningdeeplearning " go? The visual signal processing requirement alone is enough to require the power of a datacenter. And how amazing are we humans, with our highly specialized bit of wetware with a vast visual processing cortex, can do it all at 25 paces. The next step is to make a computer system do it. This is where the Cadence Tensilica Vision family of DSPs come in, specifically the Vision P6 DSP, which set new standards in neural network performance for a general-purpose imaging and computer vision. There is simply nothing faster. So you can bet that no matter where AR/VR/IR/MR/OMGBBQR may take us in the future, Cadence will likely have a hand in the pie. —Meera![]()
↧