In short, the answer is no, you can't back out if the lower court judge adds conditions that are not part of the plea agreement. Once the judge accepts your guilty plea, you can't change your mind simply because the judge didn't follow the agreed recommendation. The good news is that most judges will follow agreed recoendations because they know that both sides have put time and thought into the resolution and know the facts better than the judge. You should talk to your attorney about the judge you are assigned to for the plea and see if he/she usually follows agreed recoendations or if you should consider affidaviting the judge and asking for a different one.
Different functions are done by different entities. The Prosecutor's function is adding or removing (amending) charges - hence, you see a Felony dropped down to a lower Court. Only the Prosecuting Attorney has the ability to "change" the charges.
In your scenario, the Court's function is sentencing. The Court can not change the charge, but can sentence you to anything inside of the maximums allowed. With a Gross Misdemeanor that is anywhere up to 365 days in jail. However - regarding an agreed and recommended jail term, most judges will follow a well thought out resolution. Other condtions (the anger management, assessments etc...) is where you see Judges "freestyle" a bit and this is within their ability to do so. Hopefully this helps.
I agree with Mr. Wagnild's answer. If the judge in the lower court goes sideways on the plea deal, there will not be much you can do. That is always a risk. However, it sounds like you and your attorney have negotiated a fairly detailed arrangement in sending the case down to a lower court. Most judges will not want to throw a wrench into a complex arrangement like this. However, you should always check with your attorney about the judge who will be sentencing you in the lower court. Your attorney may have information (hopefully good information) about the judge's willingness to follow agreed plea recommendations.