You can sandpaper the primer or gesso that's on the canvas to make it smoother, but you should be careful you don't damage the actual threads or fibers of the canvas. Remember, the primer is there to seal the raw canvas and protect it from the oil paint.
Use a fine sandpaper, not a rough one that's really abrasive, and work carefully. If you're applying extra gesso, sand it between each coat.
You should also check to see if your local art store has different kinds of canvas -- some have a tighter weave (threads) than others. Canvas made from linen has the tighest weave.
But if you're after an extremely smooth surface for painting on, you should consider using hardboard rather than canvas. See Painting on Hardboard for further information on this.
