Law enforcement officers can generally arrest a person with an outstanding warrant practically anywhere.
If your unemployment case officer knows about your warrant and notifies law enforcement officers, those LE officers may show up and arrest you if they want to arrest you.
Alternatively, the LE officers may figure out that you are collecting unemployment benefits and ask the unemployment office to call you in.
Or, you may just be worrying about things that would not happen.
No one here knows anything about you and would not be in a position to opine whether your worrying is reasonable.