In 1919, one year after the end of the First World War, the British Mandate in Palestine and Transjordan began. The British, with their renowned tradition of excavating and handling archaeological finds, devoted much effort to cultivating archaeology in the land and in 1920 established the Mandatory Department of Antiquities. The department was responsible for supervising all archaeological activity in Palestine and for protecting the ancient sites.

A few years later, in 1925, James Henry Breasted, founder and director of the University of Chicago's Oriental Institute, visited Palestine. Upon arriving in Jerusalem he was surprised by the city's lack of a proper archaeological museum. Encouraged by Lord Plumer, the British High Commissioner, Breasted approached American philanthropist John D. Rockefeller, Jr. for a donation that would help realize the construction of such a museum in Jerusalem. Rockefeller agreed to contribute toward the project the sum of two million dollars - at that time an astonishing amount. A short while later, a site was chosen for the building - Kerem el-Sheik - a hill located just outside the northeastern corner of the Old City walls.

The design of the museum was assigned to Austen St. Barbe Harrison, the chief architect of the Mandatory Department of Public Works, who planned a magnificent structure, integrating in it eastern and western architectural elements. The cornerstone of the new museum was laid on June 19, 1930. Although its official name was "The Palestine Archaeological Museum," from the very outset, the building was most commonly known as the "Rockefeller Museum" after its benefactor's name. On January 13, 1938 the museum opened its doors to the public.

In May 1948, shortly before the termination of the British Mandate in Palestine, the High Commissioner founded an international board of trustees, consisting of twelve members representing scholarly institutions around the world, to be responsible for the museum's management. This board continued to function until 1966, when the museum was nationalized by King Hussein of Jordan. Just a few months later, at the close of the Six Day War in the summer of 1967, the Rockefeller Museum was transferred to Israeli hands. The management of the building and its contents was entrusted to two institutions: The Israel Museum, Jerusalem and the Israel Department of Antiquities and Museums (later renamed Israel Antiquities Authority). The former was put in charge of the exhibition halls and the latter was given responsibility for the remaining wings (storerooms, archive, library). In recent years much effort has been devoted to updating existing wall panels, to adding additional ones in English, Hebrew, and Arabic, and to the preparation of temporary exhibitions on a variety of topics.