Can someone sue me for getting diminished value of car involved in an accident?

Hello,

I was involved in a car accident in October last year where I was in fault (Two cars were involved including mine and my car hit the back of other car). My insurance company paid repair costs of both cars. However, the owner of other car now claims diminished value of his car and my insurance company is not agreeing to pay the amount he claims. The person is now threatening me to sue. What should I do?

Animesh