All in the Wrist
This technique works well when accuracy matters and when you have multiple buttons to mash. Hold your PSP and see how fast you can hit buttons with your thumb, as you would normally press them. Now loosen your grip with your right hand, holding the console tightly in your left. Let the muscles in your right hand and wrist go limp before rapidly shaking your wrist up and down. After some practice, you'll likely notice this lets you hit your buttons quicker than simply moving your thumb up and down over the button.
Rub the Button
This method doesn't actually require any mashing, but rubbing. Holding the PSP firmly with your left hand, position your right thumb over the button you want to mash and rub over the button, pressing it down and releasing it as you end the stroke. Move your thumb back to its starting position to press the button a second time. This is easiest when combined with the "All in the Wrist" method; switching the technique keeps your thumb from getting fatigued as easily.
Two-Fingered Mash
Set the PSP on your lap or a table and hold it steady with your left hand. Using your right hand's index and middle finger, position your finger tips over the button you're mashing. Press down first with either finger and quickly press the button with the other as soon as you lift the first finger off the button. This technique lets you mash longer, as your two fingers working together fatigue slower than one finger by itself.
All in the Elbow
Set your PSP down on a table or your lap and hold it in place with your left hand. Press your right index finger and thumb together, like you're pinching something. Instead of bending your wrist, keep your wrist firm and straight. Using your elbow to bend, quickly lift your index finger and thumb from the button and bring them down on it again to mash the button.
Accuracy and Precision
More important than any specific technique, accuracy and precision help determine your overall button mashing speed. If mashing just one button, use whichever technique prevents you from accidentally pressing another button instead of the target button. Practice the technique while focusing on just lifting your finger or thumb enough to release the button, keeping your finger or thumb as close to the button as possible. The smaller your movements are, the less fatigued your finger will get and the quicker you'll mash the button.