Instructions
Move your on-screen character so that he is positioned directly in front of Dogmeat. When you get close enough to Dogmeat, a list of options will appear on screen. Select the "Talk" option using the button displayed on your TV. For example, on the PlayStation 3. you talk to characters using the "X" button.
Use the directional stick on your controller to highlight the second option from the bottom of the list on screen. This option tells your follower (in this case, Dogmeat) that you no longer want them to follow you around.
Select the menu option using the appropriate button. On the PlayStation 3, you select the option with "X". On the Xbox 360, you select the option with "A". Your on-screen character will tell Dogmeat that they need to go separate ways. Dogmeat will walk around the capital wasteland but he will no longer directly follow you.