Lady Selphy
Look at the rocks in the room of the cave you're currently in and count them. The number of rocks, which is always 3, 6, 9 or 12, corresponds to a clockwise cardinal direction. 3 is east, 6 is south, 9 is west and 12 is north. Follow the direction indicated by the rocks, which leads you to a battle with Lady Selphy, and the exit to the cave.
Lax Incense
Return to the start of the cave and head east then south. This leads you to the Lax Incense, which increases your Pokemon's evasion by 5 percent and allows Wobbuffet to breed a Wynaut.
Rare Candy
Return to the start of the Lost Cave, then head east, north, south, south, east, west, south then west. The Rare Candy increases any Pokemon by one level.
Sea Incense
Go back to the start of the cave, then go east, north, south, and then north to pick up the Sea Incense. The Sea Incense is a hold item that allows Marill or Azumarill to breed an Azurill. It also buffs Water-type moves.