How can they put my picture up there without my permission???
Police records, including mugshots, are usually considered "public records" or "public domain records' -- the exact term varies from state to state. In fact, many states are actually required by law (typically a "Sunshine Act" law or the like) to keep this information available to the public. In some states, pictures and mugshots are expressly protected as a "public interest", meaning that under the law, the public is considered to have a valid legal interest in having access to your photos. As a result, in most situations, it is perfectly legal for a private website to take these publicly accessible photos and put them up on their website.
But isn't this copyright infringement?
No. Copyright in a photo exists with the person taking the photo, not the subject of the photo. You don't take your own mugshot; therefore you don't have copyright in it. Government works are also typically considered public domain as well, meaning anyone can do whatever they want with it, including put it up on a website.
But isn't this libel/slander/defamation?
Generally no. As noted above, it is a matter of public record that you were arrested for something if your mugshots are made available by the state or country that took them. If a private website republishes that information, they are simply stating the facts that you were arrested and this is your photo. Truth is a defense to defamation claims. The area where this gets somewhat sticky is if your record was administratively cleared, expunged, sealed or otherwise modified. But these cases are rather uncommon and 99% of the time, defamation would not apply, because even if the website got it wrong, you would typically need to show malice on their behalf -- an intention to harm you by posting those photos that supercedes their First Amendment rights to publish them.
What if that's not me?
If you're falsely identified as the subject of a mugshot, that's an entirely different story. Clearly something went wrong in their process, and you may have a case against them. It's extremely rare for this to happen, but it could in theory happen. If it did, consult a lawyer familiar with libel/defamation cases in your jurisdiction.
So should I pay?
Maybe, maybe not. Typically these sites charge between $100-400 to remove a mugshot. However, remember these are public records and your payment is only between you and the site you pay. So let's say there is a hypothetical mugshot site A, and a hypothetical mugshot site B with no connection to each other (other than same general industry and style). If you pay to remove your mugshot from site A, that does not mean it will be removed from site B. You'd need to pay them as well to make them remove it. This can get expensive quick, especially at the exorbitant rates these sites often charge. Whether or not you choose to pay is a personal decision, but keep in mind that if you do, you may end up paying more than you think. At the bottom of this page is a link to an article on Wired.com which outlines how the removal services work. The important thing to remember is that there are NO legal tricks to this. It's simply paying for deletion. Some might call it a scam but it's not illegal.