You can try to work with your former employer to get a good response, but your former employer does have the right to not speak about you. They may be afraid of saying the wrong thing and having you sue them. If I ever represented an employer, which I dont, I would not want them commenting on former employees to anyone that called.
A former employer is not required to give any information to a prospective employer who calls for a reference, The former employer has the right to refuse to give out any information, although most will at least confirm the start and end dates of employment. A prospective employer calling for a reference usually will not draw a negative conclusion from the refusal to give a reference, but of course a lot can be conveyed in the tone of voice of the former employer.
If you're curious as to how your former employer is handling these reference calls, you could have a friend or a private investigator call them, posing as a prospective employer.
If you feel that your former employer is giving out bad vibes about you, you could say, for example, "Feel free to call my former employer for a reference, but from what I hear, they don't give out any information about ex-employees." Better yet, if you were friendly with anyone at that doctor's office, tell the prospective employer to ask for that person directly -- but first, you should talk to that person and let them know you are using them as a reference.