You will more than likely face some liability for passing off someone else's product as your own. If you are a licensed distributor (which I assume you are because you purchase wholesale) then you will likely be in violation of your distribution agreement.
The answer to this question is for informational purposes only and does not form an attorney-client relationship.
I have read your comment to attorney Malek's post and it is correct, although your understanding of it may be a little off.
The real issue is with trademark law and if you are only replacing the label then you would be liable for passing the work off as your own, which is a violation of trademark law.
The federal trade commission statement relates to a case (and cases along the line) of Champion Spark Plug Co. v. Sanders (you can look it up on Google Scholar). The case noted that it is alright to refurbish spark plugs and leave the original company's name on the goods as long as the company doing the refurbishing has its name listed on the product and the attributes that were caused by the changes in the original product (in the case, the refurbishing) were attributable to the company making the changes to the good, not the original company. What the Champion Plugs case and the Federal Trade Commission statement is not referring to is passing off someone else's goods as your own (which you would be doing if you only changed the label). It is alright to make changes to the original goods, but you would want the goods to appear to the customer as the original good was attributable to the original manufacturer and the changes were attributable to you (in the Champion Plugs case, the original plug was attributable to Champion, whereas the refurbished characteristics were attributable to Sanders; this matches up with the perfume case that many of the other attorneys will quote). So, just removing the label and replacing it with your own would be a violation of trademark law.
Answering of your question is merely general advice and does not constitute legal advice. None of the statements or implications made by this answer creates an attorney-client relationship with the attorney answering the question. The statements made in this answer are not to be solely relied upon and you should meet with a competent attorney to discuss any concerns you may have regarding this answer.
That is classic reverse passing off. The problem is not the Textile Act, it is 15 USC 1125 and some Supreme Court opinions saying this is an illegal form of unfair competition. After all, it is a false designation of origin intended to deceive consumers into thinking you are the source of someone else's branded goods. It also deprives the brand name owner of goodwill, so it could be considered a form of theft called reverse passing off.
That you buy wholesale is wholly irrelevant.
I am not your lawyer and you are not my client. Free advice here is without recourse and any reliance thereupon is at your sole risk. This is done without compensation as a free public service. I am licensed in IL, MO, TX and I am a Reg. Pat. Atty. so advice in any other jurisdiction is strictly general advice and should be confirmed with an attorney licensed in that jurisdiction.
As my colleagues note, you may not lawfully remove the BRAND label that's affixed to the clothes you purchase. That is unlawful reverse passing off.
The FTC paragraph that you've quoted relates ONLY to the label required by the Textile and Wool Acts that are used by importers to identify the fiber content, the country of origin, and the identity of the manufacturer or another business responsible for marketing or handling the item. That is a DIFFERENT label than the one used by the manufacturer to brand the clothes.
The above response is general information ONLY and is not legal advice, does not form an attorney-client relationship, and should NOT be relied upon to take or refrain from taking any action. I am not your attorney. You should seek the advice of competent counsel before taking any action related to your inquiry.