He was afterwards appointed the prince's envoy at Paris, where he remained until the decree of Napoleon, forbidding all persons born on the left side of the Rhine to serve any other state than France, compelled him to resign his office (1811).
He then retired to Vienna, and in 1812 he took part in the attempt to excite a second insurrection against Napoleon in Tyrol. On the failure of this attempt he left Austria and joined the headquarters of the Prussian army (1813), and became a member of the board of administration for north Germany. In 1814 he was appointed administrator of the Orange principalities; and, when the Prince of Orange became king of the Netherlands, Gagern became his prime minister.
In 1815 he represented him at the Congress of Vienna, and succeeded in obtaining for the Netherlands a considerable augmentation of territory. From 1816 to 1818 he was Luxemburg envoy at the German diet, but was recalled, at the instance of Metternich, owing to his too independent advocacy of state constitutions. In 1820 he retired with a pension to his estate at Hornau, near Hochst, in Hesse-Darmstadt; but as a member of the first chamber of the states of the Grand Duchy he continued to take an active share in the promotion of measures for the welfare of his country. He retired from public life in 1848, and died at Hornau. His son, Max von Gagern, also became a politician.