It's not clear cut. It comes down to who had the last clear chance to avoid the collision. Report it to your insurance company, they're supposed to have your interests in mind (as well as their own) in their attempts to get the other driver's insurance company to accept liability.
Disclaimer: Please note that this answer does not constitute legal advice, and should not be relied on, since each state has different laws, each situation is fact specific, and it is impossible to evaluate a legal problem without a comprehensive consultation and review of all the facts and documents at issue. This answer does not create an attorney-client relationship.
You should probably pursue the matter as a small claims complaint against him. Let the judge decide the merits of both arguments and, hopefully, you will win and get a judgment against him. Neither of you will be allowed to have an attorney represent you so you will be on "equal footing". His adjuster will also not be allowed to show up and/or participate. If you prevail, it will be much more likely that his insurance carrier will just pay the claim.
A roundup of the best tips and legal advice.