In “An Immersive System with Multi-modal Human-computer Interaction” Zhao et al. argues that a multi-model human-computer system that incorporates facial behavior, body gesture, and spacial location can help decipher the intent behind users.
Zhao and his team use modern day technology for monitoring human features, for example, IBM’s Bluemix is used for speech recognition and helps them map intent behind the words a user speaks; the example given in the text is: “a user could say “hello” or some variation of it and the conversation service will map the intent as “greeting”.” They also use Kinect cameras for gesture tracking, so if a student points to something then the system could get an idea of what they’re interested in.
The system was put into action by simulating a restaurant and having people try to learn a new language by ordering a meal in Mandarin. The results of this test overwhelmingly pointed towards a success. Most students liked how they could just ask how to say something and they would get a response; they also liked how they could simply point to a menu item they didn’t understand and the computer would tell them about it, and this is just one application of the system.
The idea behind this is similar to the TED talk we watched in class the other week, where the entrepreneur made a device that would tell you about what you place on it; for example, if you were in an airport and you put a boarding pass on the device then it would tell you where your gate is. This idea of creating devices that tell you what you want to know before you even know it is extremely interesting to me and is why I study machine learning. So, if we could make a device similar to this one and put it in a classroom that could tend to kids while the teacher is busy, maybe with another child, then we would be opening up a whole new world of equality. To go into a bit more detail, it is well known that children learn at different rates and in different ways that cannot always be catered to in the classroom setting due to there only being one teacher. So, if there were a computer that could teach children, but also cater to their specific learning style, then the playing field would be far more leveled, and I think that is one of the most promising applications of AI in the short-term.