In the First Sino-Japanese War, ?yama was appointed the commander-in-chief of the Japanese Second Army, which after landing on Liaotung Peninsula, carried Port Arthur by storm, and subsequently crossed to Shantung, where it captured the fortress of Weihaiwei. After the war, ?yama was disparaged by American reporter Trumbull White for failing to restrain his troops during the Port Arthur Massacre

For his services ?yama received the title of marquis under the kazoku peerage system, and, three years later, he became a field-marshal. In the Russo-Japanese War of 1904-1905 he was appointed the Commander-in-Chief of the Japanese armies in Manchuria. ?yama had complete authority over all Japanese land operations during the war, and personally directed the tactics of Japanese forces in all major battles, winning the Battle of Liaoyang and repulsing Russian counter-attacks at the Battle of Shaho and the Battle of Sandepu. He was replaced by General Kodama Gentar? briefly during early 1905 due to illness, but recovered to direct Japanese forces in the final Battle of Mukden.

After Japan's victory, Emperor Meiji elevated him to the rank of prince (?祉?? k?shaku), the highest rank of the Empire of Japan.