There is not enough information provided to answer your question. Is there an estate open? Are you the executor? If so, then you would have legal authority to remove your brother. If not, then you have no authority to do so. You would have a right to occupy the property with him, assuming there is no one else on the title. But neither one of you would legally be able to sell the house or refinance unless and until an estate is open. If you are acting as executor, you should be represented by an attorney, to make sure that you handle things properly.
*** LEGAL DISCLAIMER I am licensed to practice law in the State of Michigan and have offices in Wayne and Ingham Counties. My practice is focused in the areas of estate planning and probate administration. I am ethically required to state that the above answer does not create an attorney/client relationship. These responses should be considered general legal education and are intended to provide general information about the question asked. Frequently, the question does not include important facts that, if known, could significantly change the answer. Information provided on this site should not be used as a substitute for competent legal advice from a licensed attorney that practices in your state. The law changes frequently and varies from state to state. If I refer to your state's laws, you should not rely on what I say; I just did a quick Internet search and found something that looked relevant that I hoped you would find helpful. You should verify and confirm any information provided with an attorney licensed in your state.
Depends. NJ has anti-eviction laws, which may or may not give your brother occupancy rights. In addition, your mom's estate plan may or may not give such rights. Assuming your brother has no occupancy rights, the personal representative of the estate (person with Surrogate's Court granted Letters Testamentary or Letters of Administration can force your brother out, but it could require a court order to get law enforcement to help. If the liens and other charges against the estate exceed the value of the estate, the family could be better off just walking away from everything, but that depends on many factors. You really need legal advice here. Lawrence Friedman, Bridgewater, NJ. Certified as an Elder Law Attorney by the ABA approved National Elder Law Foundation, former Chair NJ State Bar Association Elder and Disabilities Law Section, Member Board of Consultors of NJSBA Real Property, Trusts & Estates Law Section, Vice Chair Special Needs Law Section of National Academy of Elder Law Attorneys, and Master of Laws (L.L.M.) in Taxation from N.Y.U. School of Law.
The responses of the two other attorneys are on point. You should ascertain who was paying the bills for the property while your mother was alive. Who do you anticipate will be paying the bill for the property if your brother remains and/or if he leaves? Who is the "We " that cannot sell the house and dislikes the problems that are caused by the brother?