If the restraining order is violated, report it. With respect to a defamation claim, know that defamation lawyers are very expensive, and typically want a $5,000.00 retainer just to begin work, and without a monetary loss, you may end up spending more money than you would recover, if anything.
Contact the court that issued the restraining order, the district attorney, the police or whomever helped you get the restraining order, let them know it was violated.
"Can I press charges against them? " for defamation no. For violating the restraining order the district attorney can press charges.
You could could start a civil action against them for defamation, but it is probably not worth your time.