In 1973, he was appointed Assistant to the United States Secretary of Defense and the United States Deputy Secretary of Defense, serving under Elliot Richardson and later James R. Schlesinger. In 1975, President of the United States Richard Nixon appointed Murray as Deputy Assistant Secretary of Defense (Manpower and Reserve Affairs), a post he held until 1976. During his time at the United States Department of Defense, Murray worked on implementing a new NATO strategy; transitioning the United States Armed Forces following the ending of the Vietnam War; and the transition of the United States Armed Forces to a volunteer military.
In 1978, President Jimmy Carter appointed Murray as Deputy Assistant Secretary of Defense (International Security Affairs). During this period, he participated in Carter's negotiation of the Camp David Accords between Israel and Egypt.
President Carter nominated Murray as Under Secretary of the Navy on January 17, 1980. He was sworn in on February 7, 1980 and served as Under Secretary until September 29, 1981.
Following his stint as Under Secretary, Murray moved from public service to academia. He first joined the Naval War College in Newport, Rhode Island as Dean and Director of the College's Advanced Research Center and creator/director of the Strategic Studies Group. In 1983, he became a member of the faculty of the John F. Kennedy School of Government at Harvard University as director of the School's national security program. Murray retired from teaching in 1990.