They can be any of the above :) Many former judges in the twilight of their careers take on the job of being a "Neutral". (The word Neutral covers both Arbitrators and Mediators).
Often lawyers do that work as well as non attorneys who have specialized training in the "art" of mediation or arbitration.
There are many types of Mediation and Arbitration, but for the most part, Mediators try to help the parties come to an agreement on issues in a case and try to help both see the merits in the others cases and the weaknesses in their own. In contrast, most often Arbitrators are the "final" say and if it is binding arbitration (which is the one I think you are likely asking about) they more or less act as a judge and jury. There can be a panel of Arbitrators or just one. Either way they the advantages of the Arbitrator over a trial is that the rules of evidence are greatly relaxed and the parties save a LOT of money as opposed to litigation in a court.
This answer does not, nor is it intended to, create an attorney-client relationship or constitute attorney advertising. Rather, it is offered solely for information purposes. Since the facts of each case are different, it is critical to consult with qualified counsel with whom information can be shared and assessed under an attorney-client privilege, so that competent advice can be obtained on which you can make informed decisions.
Arbitrators and mediators are different roles. Arbitrators are basically private judges who conduct formal, private trial-like proceedings. Often retired judges may serve as arbitrators. It is not essential to have legal training; labor arbitrators may not even be lawyers; but usually arbitrators are experienced at conducting formal legal proceedings.
Mediators are persons employed to assist disputing parties to discover whether agreement may be reached. Mediators have no power to impose settlement (as contrasted with arbitrators, who have evidence submitted to them and often impose a decision which binds the parties). Good mediators understand the boundaries of their role, and will often have many years' experience with advocacy and litigation. They point out the risks of losing to the defendant, and the weak points of the case to the plaintiff, in an effort to get each party to adjust expectations about what a realistic settlement would involve.
The first two answers are good and I will just add on what it takes to become an arbitrator or mediator. Informal arbitration and mediation can occur among friends. Suppose Jane and Mary are friends who get into a dispute about money. If one of them says,"Let's ask Sue what is the right way to resolve this and we'll agree to accept what she says," and the other agrees, they have just entered into a binding arbitration agreement. If one of them says, "Let's ask Sue what she thinks," and the other agrees, Sue has become either a mediator or a non-binding arbitrator.
In many court systems there are court appointed arbitrators or mediators. For example, in Clark County, Nevada, where I practice, most lawsuits for money not involving more than $50,000 must go to non-binding arbitration before a court appointed arbitrator. I am one of many court appointed arbitrators spending maybe three hours a week on average on these arbitrations. I find it interesting to see how different lawyer approach cases. To qualify for this kind of arbitration I had to take a course, have practiced for more than eight years, and be in good standing with the bar.
As one of the other attorneys answering this question pointed out, many judges after retiring from the bench do private arbitrations or mediations and often are able to command very high fees.
I recently mediated a legal malpractice case--I was representing the plaintiff--before a retired judge, in this case Judge Bell. While a judge, Bell earned a reputation for fairness and as a mediator he has a good track record of getting cases settled. In this mediation, which was paid for by the legal malpractice insurance company Judge Bell put the two sides in separate rooms and shuttled between the two rooms. We spent a morning on this and got the case settled for $75,000. I expect that the mediation service that Judge Bell worked for billed the insurance company $3,000 to $5,000 for this. (The sides submitted confidential briefs to Judge Bell earlier but it was still very lucrative work.)
If a case is tried in federal court, a federal magistrate will conduct a settlement conference (for free) for the parties if they want it. I recently participated on behalf of my client injured in a boating accident in a settlement conference with a federal magistrate in Southern California. In this case the boater's insurance company was willing to pay its $300,000 policy limit but that was no where near enough to compensate my client for his injury. In the settlement conference the magistrate was able to get the parties to agree on an additional sum paid by the defendant personally out of his own pocket. (That rarely happens, by the way.)