Gravity
Due to the sun's immense size, it exerts a large gravitational force on all the objects in our solar system. In fact, the solar system is described as the area that is affected by the sun's gravitational field. This distance could be as far away as two light years. This force acts to either pull the objects towards the sun or to at least prevent them from exiting the solar system.
Momentum
Objects in motion tend to stay in motion. In space, there are few forces -- such as atmosphere or friction -- to slow an object down. One major force at play, however, is gravity. Since planets and other bodies are moving, they would shoot off into space in a straight line if not for the sun's gravity. The gravity field prevents the planets and other objects from escaping the solar system.
Speed.
If the sun's gravity were the only force acting upon the solar system, however, the objects would eventually be pulled directly into the sun. Its force would slow, then stop and eventually reverse the planets' momentum. Because the objects are moving so quickly, the force of gravity is strong enough to bend the their path, but not so strong as to stop them entirely. The constant interaction between the movement and the gravity keeps the objects in orbit.
Ellipses
There is one other factor that affects the orbits of objects in the solar system -- the other objects themselves. Planets have mass, and that mass exerts a gravitational pull on the other planets. It is a very small pull compared to the gravity of the sun but it's still enough to change the path of the orbit from a true circle to an ellipse.