Instructions
Turn off your PS3, and plug in one end of the optical cable into the back of the PS3. The Optical input will be located to the right of the Ethernet input.
Plug in the other end of the optical cable to your audio device input. Most often this will be your receiver.
Plug in the video cable to the PS3 and into your TV (if you have not already done so). The Optical cable is for audio only, so you will still need the video connected correctly. If you are using a connection that offers both audio and video together such as a component cable, it is OK to keep the audio cables plugged into the TV as they won't affect the optical cable settings.
Turn on the PS3 and let it load to the main menu. Select "Settings," choose "Sound Settings" and click "Audio Output Settings." Choose "Digital Out (Optical)" from the audio options, and hit the "X" button on your PS3 controller. Save these settings.