The paternal grandparents have significant rights and options. You can file for temporary guardianship of the child immediately, so that he can be properly taken care of while the mother gets her act together. I urge you to speak to an attorney about this issue, and provide some more details about your situation so that you can get the help (and your grandson can get the supervision and guidance) that you both need. Many law firms, like my own, offer free initial consultations.
Sign up to receive a 3-part series of useful information and advice about child custody law.