It's not clear cut. It comes down to who had the last clear chance to avoid the collision. Report it to your insurance company, they're supposed to have your interests in mind (as well as their own) in their attempts to get the other driver's insurance company to accept liability.
Disclaimer: Please note that this answer does not constitute legal advice, and should not be relied on, since each state has different laws, each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue. This answer does not create an attorney-client relationship.
Your other alternative would be to file an action in Small Claims Court against the driver and owner of the other vehicle. You will have to prove the other party was at fault.
You should probably pursue the matter as a small claims complaint against him. Let the judge decide the merits of both arguments and, hopefully, you will win and get a judgment against him. Neither of you will be allowed to have an attorney represent you so you will be on "equal footing". His adjuster will also not be allowed to show up and/or participate. If you prevail, it will be much more likely that his insurance carrier will just pay the claim.
Sign up to receive a 3-part series of useful information and advice about personal injury law.