On December 12, 1960, Democratic President-elect John F. Kennedy appointed Rusk Secretary of State.

As Secretary of State, Rusk believed in the use of military action to combat Communism. Despite private misgivings about the Bay of Pigs invasion, he remained noncommittal during the Executive Council meetings leading up to the attack and never opposed it outright. During the Cuban missile crisis he supported diplomatic efforts.

Early in his tenure, he had strong doubts about US intervention in Vietnam, but later his vigorous public defense of US actions in the Vietnam War made him a frequent target of anti-war protests.

Rusk also drew the ire of supporters of Israel after he let it be known that he believed the USS Liberty incident was a deliberate attack on the ship, rather than an accident.