Exploring the intersection of State Space Models and Mixture-of-Experts for low-resource environments... and wondering if they can learn to plan ahead.