I am sorry that you are going through this. Grandparent rights vary and depend upon your states statutes but regardless, the grandparents have to have ties to the children and an awful lot of advantages to change custody and your questions seems to indicate they do not.
Legal disclaimer: The response given is not intended to create, nor does it create an ongoing duty to respond to questions. The response does not form an attorney-client relationship, nor is it intended to be anything other than the educated opinion of the author. It should not be relied upon as legal advice. The response given is based upon the limited facts provided by the person asking the question. To the extent additional or different facts exist, the response might possibly change. Attorney is licensed to practice law only in the State of Massachusetts. Responses are based solely on Massachusetts law unless stated otherwise.
It is it very difficult for grandparents (or any other person) to take custody of a child away from a biological parent unless the biological parent is not a "suitable custodian", and it would depend on the circumstances. It would probably take a case of abuse or neglect by the parent with whom the child has been living.
Why would the grandparents be threatening legal action?
I would recommend consulting an attorney.
See my AVVO Legal Guides on child custody, interstate custody jurisdiction, and grandparent custody cases..
This AVVO Answer is provided for general educational purposes only. By using or participating in this site you agree and understand that there is no attorney client relationship between you and the attorney responding, and no attorney-client confidentiality. The law changes frequently, and varies from jurisdiction to jurisdiction. The information provided in this Answer is general in nature and may not apply to the factual circumstances described in your question. The applicable law and the appropriate answer may be different in the State or States where the relevant facts occurred. For a definitive answer you should seek legal advice from an attorney who (1) is licensed to practice in the state which has jurisdiction; (2) has experience in the area of law you are asking about, and (3) has been retained as your attorney for representation or consultation. Your question and the attorney’s answer may be used for promotional or educational purposes
Sign up to receive a 3-part series of useful information and advice about child custody law.