Most Water Ran Underground
Even though the Roman aqueduct is an unmistakable image, such above-ground structures comprised only a small portion of the water supply system in Rome. They were used only when it was impossible to dig channels under the ground. Such channels were easier to build and maintain, and would also keep the water cleaner. So, to ensure clean water supply, the Romans, first and foremost, kept the water away from the elements, running deep underground whenever possible. Consequently, the need for filtration was less than one might assume.
Romans Let the Water Settle
An additional step that was not technically a filtering technique, but nevertheless helped to remove debris and imperfections, was the use of collection stations along aqueducts.
As the water moved towards the cities, it would occasionally flow into, and then out of, a large pool.
In his excellent book "Roman Aqueducts and Water Supply" author Trevor Hodge explains this system as follows: "... [the water] was simply cleared by allowing the impurities to settle by gravity"
Lead Filters were Utilized at Times
Along the aqueducts, the Romans also used, albeit not very often, perforated lead filters. These were large surfaces with relatively big holes in them and worked much like strainers. It is unclear, based on the evidence at hand, whether their purpose was to make water more palatable and clean or to merely remove large objects such as wood and dead animals, which could clog the system around turns and narrow passages. The latter hypothesis appears more likely, in light of the evidence.
Sand and Charcoal Filters
In "Roman Aqueducts and Water Supply," Hodge explains that the water "could likewise be cleared by letting it flow across a bed of sand." In the archeological site in Ampurias a filtration plant was found where water percolated through a set of amphorae packed with charcoal and sand. Unfortunately, it was only possible to partially recreate the site as a result of archeological excavations and experts are still trying to uncover the precise details of the filtration techniques.
It is important to notice that these techniques would significantly slow down the flow rate of water and were therefore not suitable for use along the aqueduct/pipeline system. Such techniques were utilized when the water had reached (or almost reached) its final destination and would no longer need to attain a high flow rate to travel quickly.
Final Filtration Done at Home
Finally, the Romans also purified their water immediately prior to consumption.
Author Clemens Herschel, in "The Two Books On the Water Supply of the City of Rome," writes of water coolers that had loose discs inside, which are believed to have contained filtering materials (likely sand and charcoal). Such coolers perhaps slightly reduced the temperature of the water during hot summer months while also removing further impurities.
But the system was not perfect
Despite all of these precautions, the water consumed by the Romans was not always clean. We know that the population suffered from widespread infectious disease from time to time (likely caused or made worse by contaminated water) and the lead picked up by the water as it passed through lead pipes is also thought to have contributed to large-scale poisoning.