Learning Rock Smash
Before you can go looking for fossils you must have a Pokemon that can use the Rock Smash ability to break open the rocks. The Rock Smash ability is an HM move, which means you can use it multiple times and teach it to multiple Pokemon. To receive the Rock Smash ability, travel to Violet City then head west to the junctions of Route 36 and Route 37. Talk to the old man in the area to receive the Rock Smash HM and then teach it to one of your Pokemon.
Breaking Rocks
To break rocks, you must have a Pokemon that knows Rock Smash in your active party. Stand next to the rock and press the action button and a menu will appear asking if you want to use Rock Smash. Each rock contains either a random item, like a fossil, or a wild Pokemon. Breaking open a rock with a wild Pokemon inside will trigger a Pokemon battle with a Rock Type Pokemon, so bring Pokemon with Ice, Ground and Steel type attacks.
Fossil Locations
Although breakable rocks are located throughout the world, there are only fossils inside rocks at a few select locations. Break open the rocks in the Ruins of Alph to find a Helix Fossil and an Old Amber, which is also considered a fossil. Travel to the Cliff Cave to find a Claw Fossil within the rocks. Multiple fossils are available in both areas, however their locations within these areas are randomized, so you will likely need to break several rocks to find them.
Reviving Pokemon
Fossils can be revived back into Pokemon, allowing you to gain rare Pokemon to add to your team. Travel to Pewter City and talk to the man inside the Pewter City Museum to revive a fossil back into a Pokemon. Reviving a Helix Fossil gives you an Omanyte, reviving an Old Amber gives you an Aerodactyl and reviving a Claw Fossil gives you an Anorith. You can repeat this process as many times as you like with all of your fossils.