So, let's say it's 115 degrees outside in the desert, you've just had to push your bike 500 yards or more with a flat tire and you're a solid forty minutes (or more) from home.
Do you really want to find out at that moment that you don't have the tools you need on you?
Finding the right tools to carry at all times is somewhat challenging. If you bought a used bike, the chances are that you didn't get the original toolkit. If you bought a new bike, chances are that the toolkit that came with the bike is entirely useless, except maybe to keep your map from blowing away on a windy day. That's why used bikes usually don't have them anymore.
Above, you see the contents of the kit that came with my Speed Four. Pretty huh? With the exception of the sparkplug wrench, these tools are worse than useless. When doing maintenance, I use the equivalent tool I have in my toolbox. See that big wrench with the cheater bar next to it? That's supposed to get a nut torqued to almost 90 foot-pounds loose. Yeah, right. Here in the real world, I need a three foot breaker bar and a good heave to loosen that nut, and I'm a big guy.
So, that can easily be removed from the kit. The other wrenches and the screwdriver? Terrible. They need to be replaced with the tools I actually use when doing maintenance.
In fact, a good way to figure out what tools to carry is to make a note of the tools you actually use when doing basic maintenance on your bike. Then narrow it down to tools you can actually carry, and would be capable of using on the side of the road in a rainstorm.
So, for me, I carry a variety of proper wrenches, the stock toolkit (I mean, there's a space for it under the seat, so why not), a tire gauge and a Leatherman. I also carry a flat tire kit, a big pair of pliers - the Leatherman is handy for grabbing small things, but sometimes you need more force - and a box cutter.
What tools do you have stashed on your bike at all times?