In Washington, you could choose to not exercise your visitation rights, but if you have a parenting plan in place already with visitation spelled out in the plan, then you could be in violation of a court order for violating the plan's arrangements, and you could risk court sanctions, such as a court deciding to place limitations on you should you wish to reengage in visitation if modification of the plan ever becomes an issue.
Once a parenting plan is in place (for example, a plan that says you do not have any visits built in), it takes a lot to overturn the plan later. In Washington, there is an interest in having both parents share responsibilities with children. Should you wish to wait until your children are old enough to decide, your children will not have had the benefit of seeing you in their life until they are older, whey they might not wish to see you at all. You really might consider asking yourself what's best for your children in determining visitation.
Sign up to receive a 5-part series of useful information and advice about child custody law.