Rosersberg gets its name from the mother of its founder, the Lord High Treasurer Gabriel Bengtsson Oxenstierna. His mother came of the well-known Tre Rosor ("Three Roses") family. The Palace began to be built in 1634 and was completed four years later. It was typical of its period, with a very lofty main building and magnificent, richly decorated gables in Dutch-German Renaissance style. In front of the main building was a courtyard enclosed by three low-rise enfilades.

By the second half of the 17th century, the Renaissance Palace was felt to be out of date, and the Lord High Treasurer's son, Chancery President Bengt Oxen-stierna, had it radically modernised in the current Baroque style. For this he engaged the country's leading architect, Nicodemus Tessin the Younger. New wings were added and fitted out, among other things, with garden grottoes, which have survived down to the present day. The tall Renaissance gables of the main building were demolished and the building was given a hipped roof. A splendid colonnaded gallery survives from that time. In 1747, Rosersberg was acquired by Baron Erland Broman, and was again greatly modernised. These alterations were directed by one of the country's most prominent 18th century architects, Jean Eric Rehn.

On Broman's death in 1757, the property was acquired by the State and placed at the disposal of the young Duke Karl (later King Karl XIII) of Södermanland. This is how Rosersberg became a royal residence.Duke Karl continued the process of modernisation, still under Rehn's direction. On the death of his brother, Gustav III, in 1792, Karl became Regent for the young King Gustav IV Adolf. After 1792 the Palace acquired a number of important new interiors, typified by the Orange and Red Drawing Rooms and the Hogland Room. Stylistically, this work comes midway between the neo-classical epoch and the subsequent empire period, and the Palace interiors are often referred to as "Karl XIIIth Empire".

After 1818, Rosersberg became a summer residence of the first Bernadottes. Karl XIV Johan and his consort, Désirée, made great use of this Palace on the shores of Lake Mälaren. Karl Johan's magnificent bedchamber is one of Sweden's finest interiors from the early decades of the 19th century. Following the death of Queen Désirée in 1860, Rosersberg was handed over to the infantry and for the next 101 years it was a school of musketry. Land and parts of the Palace were transferred in the 1960s to the Civil Defence and are used today by the National Rescue Services Board.