Instructions
Go to your race's home town. Night Elves go to Darnassus, Draenai go to the Exodar, Dwarves go to Ironforge, humans go to Stormwind, Orcs go to Orgrimmar, Undead goes to the Undercity, Blood Elves go to Silvermoon City and Tauren go to Thunder Bluff.
Ask any guard where the local riding master is. Some cities have multiple vendors and the guard will mark the nearest master directly on your in-game map. Guards are abundant in each city, so you will not have trouble finding them.
Follow the new marker on your map. You have approached the correct vendor when you see "<Riding Master>" listed beneath the character's name. Right-click the vendor for a dialog box to appear, listing different steeds.
Purchase the steed of your choice. New riders can only purchase the basic steeds by default. The basic steeds are the first few listed in the box. The ones you cannot purchase are greyed out. Right-click the steed you want to purchase to add it to your inventory.
Press "B" to open your backpack.
Locate the recently purchased steed.
Right-click the steed to "use" it and add it to your list of mounts.