A deckle edge on a sheet of watercolor paper is created when the paper is made, it's the natural edge to the sheet rather than a cut edge. A deckle edge is uneven or rough (rather than a straight, cut edge) and the paper thins slightly. (For a fuller explanation, see the article What You Need to Know About Watercolor Paper.)
If you're not into making your own paper, the effect can be simulated by tearing a sheet of paper along a rough edge (you can even buy a tool for this, which looks like a seriously rough-edged ruler). Or fold a sheet in half, run your fingernail down the fold, then tear the sheet gently by hand (not against an edge). While either of these methods will create an uneven edge, the paper obviously won't thin slightly towards the edge as on a true deckle edge.
Artist Heather MacD. says she has two methods for creating a deckle edge:
1. Dry Method: Hang the edge of the paper you want to deckle just out from the table and gently thin it down with fine sandpaper by going back and forth at random along the paper’s edge. You are pulling outward, 'feathering', not sawing back and forth.
2. Wet Method: Create a deckle by using water and a thick round brush (such as an oriental brush). Run a line of water down the edge and then tear it off, slowly and carefully. When dry, you can 'feather' it using the same technique with the sandpaper, especially if the paper is very thick. You are aiming for an irregular look.