Invited Technical Talk: Building Embedded Adaptive Systems

From Anita Borg Institute Wiki

Jump to: navigation, search

Invited Technical Talk: Building Embedded Adaptive Systems

Location: Salon III

Presenter: Devika Subramanian

Over the past five years, I have been working on embedding machine learning into systems of value to society. I will present my work on building weight gain embedded adaptive systems to solve two important problems. The first is in the area of training; helping humans learn complex visualmotor tasks with strategic components by tracking their eyes, hands and brains in real-time. The second is in the arena of forecasting international conflict; tracking its evolution over time using information extracted from wire news stories. I will expose the common computational core of these problems and show an approach for designing adaptive embedded systems for a wide range of applications.

Socially Relevant Computing

She likes asking the question of "what can computing do for you?" It's amazing how people will talk to you.

"Socially relevant computing: computing for a cause, finding an answer to a question that someone cares about." An example: now working with Houston to discover how computer science can help evacuate the city.

Socially relevant computing is "computing that meets a need in some context." "It could have big societal implications"---Capital "S" Societal Computing---or on be on a smaller scale. It may be problems that exist or it may be problems we create. Who could have known we would need something like Google?

Embedded systems are systems that live in an environment that they can sense, make decisions based on those observations, and chose actions in response to that. Cruise control, UPC scanners, Amazon.com's website. "Observation driven, task-specific decision making."

Embedded systems are very popular, and "we build them the classical computer science way." Go get the specifications, design it, build it, go home. This methodology works extremely well except in three situations (which may make up 80% of the real world).

"The specifications for the system can change." Eg - elevator systems. You have to know where the elevator will be needed. Usually this is based on some assumptions of how the building will be used. The usage of the building changes with time, so algorithms that are optimized to the old usage no longer work. "The funny part is that the elevator actually knows what is going on in the building." It has weight sensors, etc, but can't modify its own behaviour. To solve this we want "Adaptive, embedded systems."

"A one-size-fits-all solution is not appropriate" [couldn't keep up]

So rather than the traditional computer science approach, we need to build adaptive embedded systems. Machine learning is a mature field with a plethora of algorithms, but they are all built out of context. "How do we design systems where we don't retrofit these learning algorithms in there, put them as an afterthought."

"My dream is to have all pieces of software that we use on an everyday basis . . . to have adaptation built in." The system should "calculate decisions on the basis of learned models of the systems"


After Devika finished her PhD, she shifted from designing theoretically beautiful and useless methods to finding large-scale real world, socially relevant problems. She presents two case-studies of building adaptive embedded systems. They are in very different domains---it is not about pointing to a single example of how the machine learning algorithms have been retrofit into the traditional computer science approach. It is about a mindset that can be applied to any problem.

First case-study - tracking human learning on complex visual-motor tasks. Second - predicting the evolution of international conflict.

"Submarine School 101"' There's a very difficult game that is used to train submarine operators and more importantly make them drop out.

Questions: Why is it hard? How come people can't do perfectly? How do we affect learning?

  • POMDP with 10^14 state space and 153 actions.
  • "Challenging for both machines and humans"
  • Tried RL "for a period of time I thought it was the solution to everything"

Quick intro to RL "learner interacting with a task". See above link or slides for details.

Problems with application - state space too large, credit assignment (knowing what actions are to blame or praise for the outcome) very difficult

By creating some new RL methods and reducing the state space, built a successful learner. Machines are very good, almost 100%. "Machine learners are not nearly as effective as humans." Within the first 500 episodes, which is what the human players get, humans are better.

  • "Humans cannot introspect on this game." "They'll give us these elaborate explanations that are completely untrue."
  • Why so many games? Because there is a very skewed distribution over states. "There are some states with a very low frequency where the choice of action maters."
  • 'We could engineer it so that all states are evenly distributed and then humans could learn it faster'
  • Machine learning illustrated how the task was hard, and through working on this application the team made contributions to RL theory


Another application: Tracking human learning by learning a map from what the human is sensing to the actions they are choosing.

  • The number of possible sensor configurations is ridiculously huge, but an individual only sees a few. So a direct map is possible. "Get your data raw and build models out of raw data. When you interpret it you bring a lot of assumptions into the data"
  • Calculated the KL divergence between the distributions of strategies over windows of time
  • The strategy derivative is very chaotic, not a steady march. "Very, very discrete process."
  • So humans have distinct learning phases "strategy acquisition phase and visual-motor tuning phase." "Every successful learner looks like this. The inter-peak difference is almost the same across humans."
  • Unsuccessful learners have two styles - one never has a shift, just little refinements
  • The other kind. Very high strategy peaks, but changing too quickly. "Adopting and abandoning them" too quickly. Never giving time This is something that teachers "ought to recognize"


"How humans learn" [need info from slide]


Next question: "Can we create behaviourally equivalent models?"

  • Observe an individual for 100 games, use locally weighted regression on the training.
  • "The strawman beat our fancy-schmancy algorithm right out of the water."
  • "Always try the strawman"'
  • "Simple models from objective low-level data" is better than complex models

Next question: is this a physically real effect? The model shows it and matches behavioural data. What's really going on?

  • Joined with neuroscience, looking at how this task was learned.
  • Combined EEG readings of the activity of a human with the learning the task on a complicated version of the game.
  • Front-to-back connections really change across learning and for different learning
  • Physical support for what the computation shows.


Ran out of time for the second example.

Retrieved from "http://community.anitaborg.org/wiki/index.php/Invited_Technical_Talk:_Building_Embedded_Adaptive_Systems"
Personal tools