You have the right to make decisions for your child on who sees the child and on what conditions. It is unclear as to whether you were married. If you are divorced, they have no statutory basis to go for any placement. If you weren't, they can ask for visitation under the grandparents rights statute; however, they have to show that what you think is best for the child is not in the child's best interest (and poverty is not an allowable reason). I think that allowing the child to be in a car with an unlicensed driver and taking the child out of state without permission, would make the court question their judgment.
In accordance with the Avvo community guidelines, this communication does not constitute "legal advice", nor does it form an attorney-client relationship. You should seek counsel in your geographic area regarding any specific questions.
I guess I see it differently. Grandparents have rights under state statute when they have had a parental role in the child's life. So, if that is true, then it doesn't matter if you were married or not, or who did what and so on. The court protects existing parental rights whether it be for grandparents or parents (married or otherwise).
You should talk to an attorney face to face, and let them ask the questions they need to figure this out.
Good luck with your situation.
Both of the above answers are good, but some more thoughts: Yes, if you do not want the grandparents to obtain rights to the child, their chances for success decline if they are no longer seeing the child/part of the child's life.
On the other hand, they are more likely to sue for grandparents' rights if you do not allow them access to the child.
It is a Catch 22.