Multi-armed bandit (MAB) problems, typically modeled as Markov decision processes (MDPs), exemplify the exploration vs. exploitation tradeoff. An area that has motivated theoretical research in MAB designs is the study of clinical trials, where the application of such designs has the potential to significantly improve patient outcomes and reduce time-to-market. However, such designs have limited real-world application because of computational barriers that render exact approaches to solving MDPs impractical. We discuss the how adaptive designs can revolutionize clinical trials and present a novel approximation approach that allow for a computationally efficient implementation. We demonstrate the strength of our proposed approach through a retrospective implementation on a recently conducted phase 3 clinical trial.