A simple guide to partial observability

This guide provides an introduction to hidden Markov Models and draws heavily from the excellent tutorial paper written by Rabiner (1989). I have attempted to provide intuition into the various algorithms by using some simple fully worked examples to hopefully illuminate the mathematical descriptions provided by Rabiner. It is hoped the explanation of the concepts is sufficient to help an otherwise uninformed reader to understand later descriptions (such as Baum-Welch algorithm) where worked examples are not provided. The second part of this guide aims to explain the relationship between Hidden Markov Models (HMMs) and Partially Observable Decision Problems (POMDPs). While the primary concern in HMMs is to learn a good model, the addition of actions - the ability to make decisions - in POMDPs adds the additional concern of learning which action to select. These two problems are related to POMDPs which add the ability to learn a model and the ability to learn which action to select. Some systems address one of these concerns exclusively, others attempt to address both simultaneously. [Introduction]