Clik here to view.
People are always forgetting names. That’s because, at least in part, names are arbitrary. A name, in and of itself, doesn’t offer much context. And contextual associations are a big part of how humans form and access memories—the way the smell of pine trees might stir a memory of summer camp, or how hearing the hook of “Sweet Caroline” might transport you momentarily to Fenway Park.
A truly personalized search engine could be so useful this way. Imagine being able to google the ordinary things that slip your mind: “Where did I leave my sunglasses?” or “Wait, what were we just talking about?”
People have, at times, lamented the way the Internet can be a ruiner of ordinary mysteries. Instead of idly wondering about something youknow is there in your brain somewhere—Quick! Hum the theme song from Alf!—and letting it inexplicably pop back into your consciousness in time (or not), the answers are always at our fingertips. (Yup, that’s the one.)
If it were possible to dip into the human mind the way people seek and retrieve information online, a sense of curiosity would, I am sure, remain intact—it’d just be redirected. In some cases finding an answer means moving on to the next thing; in others, it means an opportunity to go even deeper.
“Human memory is not the same as computer memory,” said James Kozloski, an inventor at IBM who focuses on computational and applied neuroscience. “We don’t have pointers. We don’t have addresses where we can just look up the data we need.”
Kozloski wants to change that. He recently filed a patent for technology that, in the simplest terms, will help finish your sentences for you. Like autocomplete for your voice, the system is a model of human memory that could be embedded in a device and offer prompts when necessary. It would use a combination of surveillance, machine learning, and Bayesian inference—a kind of predictive modeling—to recognize when a person has forgotten something, then provide the missing information.
“The idea is quite simple,” Kozloski told me. “You monitor an individual's context, whether it’s what they’re saying or what they’re doing ... and you predict what comes next.”
The monitoring could be done in many of the ways that people are already using sensors today. It might involve Fitbit-like wearables; movement trackers like the ones smart thermostats use to determine when a person walks from one room to the next; and WiFi-connected microphones like the ones the newfangled Barbie dolls have so they can listen and reply to children. Which is to say, if you’re unsettled by the notion of a future in which omnipresent computers are watching you and listening to you: That future is already here.
A cognitive assistant would probably have to draw on short-range wireless technologies that could pair with sensors to figure out exactly what a person’s doing: distinguishing, for example, between the arm movement you make when you brush your teeth versus how your arm moves when you’re dicing garlic.
If you can get past the creepiness factor, a cognitive interface like the one Kozloski envisions could theoretically be useful for anyone, but he sees specific applications for people as they get older—and especially for those who suffer from diseases like Alzheimer’s. “The loss of ability to access memory in the moment is the beginning of the breakdown of normal cognitive function: the ability of individuals to interact with others, take care of themselves, clothe themselves, cook meals,” he said.
Such a system could help caregivers track how people are doing over time—are they forgetting important tasks more frequently?—and “perhaps prevent side effects of what are otherwise sort of innocuous episodes of forgetting,” Kozloski said. “Like getting confused, getting agitated, then putting myself at a greater risk.”
Imagine for example, if your cognitive assistant knew that when you dial a certain person’s phone number—your niece, let's say—it should also remind you of the name of her husband. The system might also know that, because of the time of day when you’re calling, the husband is more likely to pick up the phone. Or that, by checking a calendar, it happens to be his birthday.
“All of that context becomes the basis for inference as to what name should be spoken when they pick up the phone,” Kozloski said.
That information—Paige’s husband’s name is Teddy. And it’s his birthday!— could come through an earpiece or mobile device. Or it could be part of a larger—and, frankly, more Hal 9000-esque—home integration. “It could be just the speakers in your house that are primed and ready to advise, literally waiting in the wings to assist,” Kozloski said.
Waiting, but not always jumping in. That’s key.
“It would be very annoying if it were continually interrupting you,” Kozloski said. So the system would use basic pause-detection, reinforced by machine learning as a way to understand the cadence and rhythm of a person’s speech and behavioral patterns. Once a cognitive assistant knows a person’s typical morning routine, for example, it can figure out whether something is amiss. Then it can decide whether and how to help.
“An individual’s behaviors are mapped to a unique activity such as ‘putting socks on,’ or ‘making the bed,’ or ‘brushing teeth,’ or ‘chopping onions,’" Kozloski said. “All of those events can form a sort of Markov network of probabilities where we can predict with some degree of certainty, and measure the likelihood [that what’s happening] isn’t normal for that person.”
The technology would also need human feedback, either from the person using it or a caregiver, to confirm and improve the accuracy of what the machine learns over time. “The opportunity to personalize this to individual cadence or idiosyncrasies,” Kozloski said “to allow a caregiver to augment that learning such that you can disambiguate whether an individual is confused or just sleepy. All of that is in the realm of the patent.”
Which means eventually, people may not even find themselves wishing they could Google their own memories. They won’t have to.
Image may be NSFW.Clik here to view.