Instructions
Press the triangle button on your PS3 controller to open the PS3 Control Panel.
Select "AV Settings" from the Control Panel.
Scroll down to "Mosquito Noise Reduction" and choose the "Enabled" option.
Save the new setting and play the desired game. The edges on the picture should no longer look rough.