It is difficult to advise you without seeing the visitation order.
If the mother was not going to be there, of course you could drop the child off without a problelm.
Because the mother and probably her boyfriend will be there, you have every right to be concerned. I would recommend you wait until you get clarification from an attorney. Tell the grandmother if you get the OK from an attorney, she can spend the day after Christmas or Christmas Eve with her.
Sign up to receive a 3-part series of useful information and advice about child custody law.