I use an uneven sandy blend. I get my dirt primarily out of the concrete gutter-space near intersections and in neighborhoods. I find it makes for a great random mix of sandy and gravelly particles. I highly recommend it. You get rocks, sand, ground up particles of concrete, brick, etc. Once it dries after heavy rains you can sometimes find great drifts of it. I just filter it through a screen to get rid of any rocks that are big enough that I wouldn't want them randomly in the dirt mix, and then pour the remainder back and forth between a couple plastic cups in a light breeze to let the fine silty dust get mostly blown out.
I mix it with a bit of plain sand from any local hardware store or nursery for a sortof baseline. Then I just glue it down, paint, and sprinkle a few patches of scorched static grass and it looks great.
The "natural foraging" method also nets you a good collection of irregular little rocks in the process to put on bases and terrain. Also, for big chunks of rock, if you don't like the look of cork, you can buy a small package of slate tiles and smack em up into little pieces. A couple of whacks with a hammer gets the average tile started, and just dropping or strongly tapping the pieces on a hard surface (like a garage floor) can get them shaped and fractured into manageable, natural looking chunks.