A Mobile-Cloud Collaborative Approach for Context-Aware Blind Navigation
From Anita Borg Institute Wiki
Authors: Pelin Angin, Bharah Bhargava (Purdue University) and Sumi Helal (University of Florida)
Contents |
Challenges for people with visual impairments
- Indoor and outdoor navigation: quiet cars threaten people who base on sound to avoid accidents; in a lot of countries they cannot navigate independently because of non-adapted transportation means.
- Advanced technology: new interfaces for devices are not designed to adapt to visual impairments.
- Current technology is either high cost or requires infrastructure that is not ubiquitous.
- Demographics: 314 Million visually impaired people, 45 million blind.; 82% are 50 or older, thus it is hard for them to adapt to new technology.
Goals of this work
Build a system that can help blind people navigate - address the necessity of real-time guidance, portability, power limitations, appropriate interface (some people may also be hearing impaired, cannot use speech based navigation), yet maintain privacy and low-cost, minimal dependence on infrastructure.
Partners and discussions:
- Cary Supalo: Founder of Independence Science LLC ([1])
- T.V. Raman: Eyes-Free project at Google (speech-free Android app)
- American Council of the Blind of Indiana State Convention, Oct 2009
- Miami Lighthouse Organization
Examples of Mobility Requirements: avoid obstacles, walk in right direction, cross the road safely, know when you reached the destination, know if this is the right bus/train station etc.
- Outdoor Navigation: finding curbs even in snow, use public transportation, interpret traffic patterns (lights).
- Indoor Navigation: find stairs, elevator, restrooms, finding an electronic store ina shopping mall and the cheapest TV in it!
- Obstacle Avoidance: low & high / overhanging objects.
- Recognizing people / objects.
- Loadstone GPS [2], [3];
- Indoor: InfoGrid (RFID based), Jerusalem College of Technology System (local infrared beams); talking signs.
- Obstacle avoidance: RADAR/LIDAR, Kay's Sonic glasses ([4])
Proposed System Architecture
- Locally: mobile navigational awareness server / smart phone ;
- On the cloud side: webservices that the smartphone communicates to; GPS, other data; E.g. outdoor navigation with Google maps; "micello" is mapping indoor locations, shopping malls etc [5].
- Use wifi-based location tracking for indoor navigation wherever GPS does not work.
- Text-to-speech to communicate response to the user; speech recognition using the cloud
- Object recognition - Selectin software etc
- Obstacle avoidance: "time-of-flight camera" technology; also capture shapes of objects.
- Remote vision: another person in front of laptop to aid the blind person; video frames transmitted to the cloud to guide the person.
- Advantages of this system:
- Android platform advantages: supports multitasking; e.g. for route planning you need a GPS location service running in the background as well bus-schedule service interrogation etc.
- Advantages of Mobile-Cloud Collaborative Approach: open architecture, extensible, lightweight, computational power in the cloud instead of the mobile device itself, wealth of context-relevant information resources, minimal reliance on local infrastructure.
Practical Applicaton: Traffic Lights Detection
- Also useful to color blind people, autonomous ground vehicles, careless drivers;
- Fast image processing for locating lights and detecting their status;
- Some current approaches use a full computer, sacrificing portability to performance.
The mobile cloud collaborative traffic lights detector (android, google maps, Amazon Elastic compute cloud machine instance) uses AdaBoost to train the traffic light detector from images.
- Even with high resolution levels of the images, response time is still < 660ms (real-time);
- Tthe downside is that it requires continuous recording; this can be enhanced to record only when reaching an intersection (GPS).
Work in progress
There are current efforts to address privacy, robust object/obstacle recognition algorithms, options on placing the cameras (eye-level on glasses). Class: CS 590 Cloud Systems for Blind and Hearing Impaired Fall 2010.
- Ongoing work on Privacy: Active Bundles - encapsulates the sensitive data with metadata that has control access and dissemination policies; checks the trust level of servers to ensure all sensitive data is protected.
- Person Recognition using Multiple cues: use social information for that person. ([6])
- Collective Object Classification in Complex Scenes: LabelMe dataset MIT. Lessons learned:
- relational learning with multiple boosted detectors for object categorization
- multiple detectors in parallel, class label fixing based on confidence; more accurate than AdaBoost alone, higher recall than classic collective classification, miminal loss of recall w.r.t AdaBoost.
Q&A
- Some countries are not 'mapped' onto GoogleMaps; you cannot get the street names or path finding algorithms from the cloud. Yes, this is one of the concerns; you can still use a local GPS navigation system and rely on that.
- How do you know which apps to trust, for this purpose? There needs to be an organization/system controlling and rating the integrity and robustness of the applications.
- Is the solution reliable in areas with low-bandwith (rural, other countries); The system highly depends on high-bandwith/3G network.
- Have you considered using ultrasound for object recognition. Yes, we tried a combination of other technologies, but most of them would have made the system non-portable.
- What about the battery life? Audience suggestion: GPS on Android contacts the server at regular intervals; if you detect the same location over time (person is standing still or sleeping), you can increase the intervals of polling for both GPS and Image processing.
