Red Chest No. 1
In New Haven, directly to the right of the save point, follow the stairs until you reach the chest on the roof of a building.
Silver Chest No. 1
In New Haven, directly left of the save post, go up the stairs and cross a wood plank. Jump to the next building and go around the corner.
Red Chest No. 2
In New Haven, from Silver Chest No. 1, look for a red tarp and jump to it. Turn around and find the Red Chest and jump to it.
Silver Chest No. 2
In New Haven, complete the Claptrap Rescue mission, then find a little shed across from the Bounty Board. You have to crouch to find it.
Silver Chest No. 3
In New Haven, from the save post go up the stairs where the dollar sign and arrow are. Jump onto the washing machine, then to the roof with the large antenna.
Rider Sniper Rifle
In New Haven, find a blue shack near a repair kit. Walk right into the door and you should go through it. Take the elevator down and kill the enemies to collect this sniper rifle designed by the developers of the game, Gearbox.