Your having invited the restrained person into your home was not prudent but it also was not a violation of the restraining order. He is the one who is prohibited from contact, not you. You should report this incident to the police and refrain from initiating contact in the future for your own safety.
Your ex cannot press charges; only the DA can. That is not likely to happen in this case since you have the protective order, not him.