When someone refuses to offer testimony under the Fifth Amendment privilege it actually means they assert that privilege to remain silent. In order to do this, they must take the stand and assert their Fifth Amendment Privilege in front of the jury and the Judge must pass upon the issue and allow them to assert their privilege to remain silent due to the fact that whatever they say may tend to incriminate them. This is the salient issue, whether by speaking they might place themselves in jeopardy of admitting something that could cause them to be convicted later. Whenever this happens in a courtroom, whether its for a witness or the Defendant or a prosecution witness, the Judge will usually have an attorney present for that witness so that the wording of the Privilege can be properly worded. In the case of the Defendant, the attorney representing him/her will usually advise the Judge ahead of time that his client will be testifying but may need to assert his/her Fifth Amendment right to remain silent at some point. When that point arrives (usually in the middle of cross-examination) the attorney may advise the court that its about to assert the privilege and then do so when it actually arises. The court will then rule on the matter and the trial testimony continues from there until its completed.