We built a belief maintenance and update system to track user goals during interaction. We learnt a policy to select optimal actions based on the belief. We developed Dynamic Probabilistic Ontology Trees (POT), a new probabilistic model to track dialog state. Our model captured both the user goal and the history of user dialog acts using a unified Bayesian Network. We performed efficient inference using a form of blocked Gibbs sampling designed to exploit the structure of the model.

Later, we combined this DPOT semantic belief tracker for categorical concepts with a kernel density estimator that incorporated landmark evidence from multiple turns and landmark hypotheses, into a posterior probability over candidate locations. Used a deterministic policy to select actions based on the belief. Our system was demonstrated via android app.

We also built a hybrid system to send text messages by voice, where we used Google Speech API to recognize general messages. We used Nuance with our own language model as a fallback technique when Google Speech confidence was low, such as when there were corrections starting with No.