What you haven't said is why you are seeking to avoid mediation. You are pro se, so you are not accruing wasted attorney's fees.
If you really don't want to mediate a settlement, simply show up to the mediation and allow the mediator to provide their opening statement. Once that has occurred, inform your mediator that you do not feel comfortable continuing. Good faith at mediation is accomplished by showing up and listening to the opening remarks of the mediator.
Now, is this the best way to handle all family law matters? Absolutely not. Would the above resolve your unwanted mediation issue? Yes.
A mediation is non binding and cannot compel you to do anything. I believe it can only help in that you can express your concerns in an open forum. Plus its a way to voice positions and see what the other sides positions are before your hearing. If you dont like what you hear that ok because its non binding. Only through communication can you resolve your issues. Dont leave it up to a judge to determine the welfare of your child, unless mediation does help. Remember its up to both of you to foster a civilized long term resolution for your child and for yourself. Good luck.