What you did not was whether or not this was court ordered payment of attorney fees as a result of a plea bargain in a criminal case. If that were the case and it was a condition of your probation, you would be violating the terms and conditions of your probation. That could mean jail time unless you are indigent. You can not be jailed for indigency.
However, if it was related to a contract you signed with an attorney, a civil law suit could be brought against you for the breach of the contract with the attorney. Your wages can't be garnished in Texas but things can be seized if they obtain a judgment against you.
There are many things that could happen and "worst case" depends on your point of view. Your lawyer could withdraw in the middle of representing you. Without saying a negative word about you, the fact that the lawyer withdrew implies negativity your way to the judge and to the prosecutor. You could be sued. You could arguably be charged with theft of services depending on how it happens.
It all depends on what your circumstances are. If you entered into a retainer agreement you are contractually obligated to pay. If it's a part of your probation then you risk defying the court's order. End of day, you're obligated to pay what you agreed to pay.