I have physical custody of my 2 children. Do I have legal custody too?

I live in OR.
My kids dad just started paying child support a year ago. He wants to be involved in the kids lives now. But he had sexual relations with a 15 year old until she was 17. I do not want to have joint custody because he IS a pedophile, even if he has not gone to court because the cops and Child protective Services ignored it. Can I move out of the state if it is within the best interest of the kids and myself?