I don't think you can since the certificate is already issued at that point. What you would need to do is get your name changed through your state. Of course, this will take time and cost you more money. Good luck.
This information is intended as general information only. This communication does not establish an attorney-client relationship between me and the asker.
If your Oath of Allegiance is at a naturalization ceremony before a judge, you can make this request before the judge. The judge may or may not grant your request.
If your swearing in ceremony is not before a judge, you will have to have your name changed through the prescribed method of your state.
Ask the official at the ceremony again and request cancellation of the name change. They may be able to reschedule your ceremony and have a new certificate for you.
My firm does legal name change as well as immigration. If they will not process the name change, give us a call and we will be happy to handle.
This advice was intended to be general in nature and not to be taken as a legal opinion or legal advice and was not deemed to create an attorney-client relationship