Sequentially-Controlled Interactive Multi-Particle Flow-Maps for Online Feedback-Driven Search
Teaching AI to explore broadly and learn what humans actually want
When AI systems learn from human feedback given one step at a time, they tend to get stuck exploring a narrow corner of what's possible instead of finding the best solutions across the full space of options. This paper introduces IMPFM, a method that uses multiple interacting particles (candidate solutions) guided by flow maps to explore widely while learning from sequential feedback, preventing the system from overshooting toward extreme or unhelpful outcomes.
Most AI alignment methods today work well only when preferences are already known, or they chase narrow local optima that don't match what users actually want. This approach enables systems to discover genuinely diverse, high-quality solutions even when human preferences emerge gradually through interaction—making AI assistants and recommendation systems more useful and less prone to gaming metrics in unexpected ways.