Shirt answer is yes. Event hough he gave you permission, the Georgia courts would likely find that you drove with negligence and hold you responsible. Even more to be concerned about.... His insurance company could settle with the other driver and file suit against you for those damages as well.
Please don't consider this to be an atty-client relationship or a legal opinion. Please seek the advice of a licensed lawyer in your area.
Yes, you are responsible for damage to his car resulting from the "grazing;" you are also responsible for the damage to the other vehicle however, his insurance company, pursuant to the liability policy, will pay for that damage.
Sign up to receive a 3-part series of useful information and advice about personal injury law.