You can press f to pause the game and skip ahead one frame. Repeatedly pressing f lets you go ahead one frame at a time. This works on mobile too if you bring up the on-screen keyboard.

Going backwards one frame is not possible, but on PC you can go back to the last point you drew anything with ctrl+z. Storing the simulation state every frame would be impossible, as often it would use ridiculous amounts of RAM.