You are the parent and you are in the drivers seat. The Judge could give the grandparents visitation, but usually that is up to you as the parent. If you cannot get an attorney, this will be tough on you, but you just have to do the best you can.
If you can afford an attorney, even if only to draft the initial motion papers, you should find the money.
Sign up to receive a 5-part series of useful information and advice about child custody law.