The court's can approach this in two ways: 1) Grandparents have no right to visitation and can see the child when visitation is with the non-objecting parent. 2) because of the nature of the relationship--your previous custody--if it was a court-ordered guardianship--that it might be in the best interest of the child to have a relationship [i.e. visitation] with you. See an attorney for assistance.
Disclaimer: Please note that this answer does not constitute legal advice, and should not be relied on as such. This answer is offered for informational purposes only. It is not offered as, and does not constitute, legal advice. Laws vary widely from state to state and even county by county. You should rely only on the advice given to you during a personal consultation by a local attorney who is thoroughly familiar with state laws, local rules and the area of practice in which your concern lies. Each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue. This answer does not create an attorney-client relationship.