A gradient is a gradual change from one thing to something else. It is easy to understand this concept by considering the colors black and white. To create a gradient between them, we begin with the color black. Slowly the color changes across the picture from black to dark gray. The gray becomes lighter and lighter, until finally it turns into white.
A pressure gradient starts with a high pressure. Slowly the pressure exerted by the atmosphere becomes less and less until it is very low. As we move further away from the area where the center of the high pressure is located, we travel through that pressure gradient, reaching the low pressure.
As we have discussed earlier, wind is caused by the gases of the atmosphere as they move from one place to another in an attempt to equalize pressure. Thus, wind often blows outward from a high pressure location towards lower pressures. Free from other factors, it would always behave this way.