The simple answer is to avoid touching it. If you want to work over the top, then the best solution is to work 'fat over lean' starting with harder pastels (Rembrandt, Rowney for example) and working your way through to the softest (Schmincke or Unison).
You can also use fixative to 'fix' pastel -- just keep in mind that fixative darkens pastels, so use lighter tones in expectation of this effect, and remember to use it sparingly.

