Diffusion steering via reinforcement learning

Diffusion models are famously bad at one specific, visible thing: rendering legible text inside generated images. Signs come out as alphabet soup. With the Data Science Society’s research arm, I’m implementing diffusion steering via reinforcement learning to improve text-in-image generation, using a reward signal to nudge the sampling process toward outputs where the words actually read.

The appeal of steering is that it works at inference time: instead of retraining an enormous model, you guide the one you have. The open questions (what to reward, how to keep the guidance from degrading everything else in the image) are what make it research rather than engineering.

This is ongoing work, and the most fun kind: a problem you can explain to anyone in one sentence, attached to methods that go as deep as you’re willing to follow.