The Gemarah is binding on all Jews because that was the last work that was accepted by all Jews as binding. Since then, as Jews spread around the world,
individuals, the remnants whom God called, would gather in each city and country, occupy themselves in Torah study, and [devote themselves] to understanding the texts of the Sages and learning the path of judgment from them.
Every court that was established after the conclusion of the Talmud, regardless of the country in which it was established, issued decrees, enacted ordinances, and established customs for the people of that country - or those of several countries. These practices, however, were not accepted throughout the Jewish people, because of the distance between [their different] settlements and the disruption of communication [between them].
So after the writing of the Gemara, there was no more "final" work binding on all Jewry. However, within individual communities there were works whose rulings were considered final.
In the Gemara there is a contradiction between a Mishna and a Braisa. The Mishna says that if a judge made an incorrect ruling and then realized that he made a mistake, the ruling is reversed and the winner must pay back the money to the loser. However, the Braisa says that the winner keeps the money but the judge must pay from his own pocket. The Gemara says several resolutions, and one of them is that the mishna is referring to a mistake in a "dvar mishna" (a rule mentioned in the Mishna) while the Braisa is referring to a mistake "bshikul hadaas" (in a judgement call). The Gemara goes on to say that one who made a mistake in a ruling of Rav Chiya and Rav Oshiya, Rav and Shmuel, and Rav Sheishes and Rav Asi (all of whom were sages from the time of the Gemarah and not the Mishna) also is considered as to making a mistake in a "dvar mishna". The Gemarah goes on to ask "What is a mistake in 'Shikul Hadaas'?" and the Gemarah answers "If there is an argument between two Tanayim or two Amorayim and the argument ("Sugya") seems to go in favor of one of them and the judge ruled like the other".
So the Gemara defines two types of mistakes: "A mistake in a judgement call" which means that since the opinion like whom the judge ruled was not completely rejected, the loser can tell the judge "I hold like that opinion, and Hamotzei Mchaveiro Alav Haraya ('Bring proof that I am wrong and I will return the money')". Since that opinion is technically a legitimate opinion the winner doesn't have to give back the money.
However, the second type of mistake, a "mistake against a mishna" is not a legitimate ruling. It is a complete mistake without any foundation and the winner must give back the money. This type of mistake is not just when a judge ruled against a mishna, but when he ruled like a Tanna whose opinion was rejected from Halacha (like Beis Shammai).
As mentioned above, since the sealing of the Gemara, no later authority has the right to argue on it or add anything to it. Moreover, in the time of the Rishonim, the words of the Gaonim were considered as binding as a "dvar mishna".
Generally what comes out of all of this is that:
- One cannot disagree with the Gemarah
- One can argue on a Gaon, though only with very strong proof, and if one ruled against one without proog, he is not considered to have made a psak din.
- One has to lchatchila rule according to the traditions in his community. In some communities (Yemenites), the poskim generally rule like the Rambam. Other(Sfardim) rule (generally) like the Shulchan Aruch of the Beis Yosef. Others (Ashkenazi) rule (in general) like the Rama. Some communities (like Chabad), rule like the Shulchan Aruch Harav. Some (the Yeshivishe velt, as well as most of the "chareidi" circles) rule like the Mishna Brura.
This is another reason why being a posek is more than just learning the materials.