Short answer: Probably yes, but the dealer is probably going to try and charge you for the days that you used the car and a fee for the mileage.
It sounds like you entered into a spot-delivery agreement, and this sales tactic can be an deceptive, unfair and abusive for car buyers. Also, consider that your credit score may have been damaged because the dealer had you credit pulled by multiple lenders trying to find a source of financing. Each one of the credit inquiries lowers your score, making it more difficult to finance another car at another dealer, for example.
You should consult with an auto fraud attorney in your area to review your documents, and weigh the alternatives. You can find one here: www.naca.net
I am an attorney who is only licensed in the State of Florida. My answer is general legal advice based upon what I perceive your question to be, and should not be relied upon because every person's facts and circumstances are unique, and because specific laws vary from state to state. To completely evaluate a legal issue requires reviewing and evaluating all relevant facts, applicable laws and other information. My answer does not create an attorney-client relationship, and offered for informational purposes only.
Yes, you are obligated to return the car. Maybe the auto dealer gave you their own in-house financing but could not transfer your loan for reselling to some another lender. This happens all the time. How many days you used? Probably a pro rata payment has to be made. You can fight this and tell them, it is not your fault. You must have signed long series of papaers and had hectic negotiation with the sale people in quick and long succession (Usually, the guru come later with thick stack of papers, acts like good cops, bad cops). You may not see significant impact on your credit rating, and this is not even fraudulent activities. I believe you must have signed some disclaimer to that effect.
Only see a licensed attorney before you make any decision. This answer may not be perfect in any given situation. However, more fact may be required by your local attorney