I am not licensed in WV, so I may be completely ignorant, but what legal effect do you think a life time diary would or should have? Is this a legal document in WV? Because if it is not, that may explain the absence of answers to your question. No one may know what you are talking about.
DISCLAIMER: The forgoing comment is for general educational purposes only, and is not legal advice upon which the reader may rely as the commenter has no actual knowledge of the facts of the case, has not interviewed persons or examined evidence, and has not researched the applicable law. The comment is based only on the facts provided, which are extremely limited, and may or may not be true. Complete defenses may prevent the success of any claim. Competent legal advice should always be obtained before taking any legal action or filing suit. Readers employ any information provided herein at their own risk.
I will take a shot at this, although as others have said "life time diary" means nothing.
You first have to tell us who owns the property now. Your question says you and your brother do, but then refers to something your parents set up before they were divorced. Well, if they set up anything, it would mean they had to own the property for whatever they did to have any legal effect.
If you mean a "life estate" and that a life estate was given to your father, what that means is that your father has the right to live on the property for as long as he lives, regardless of who owns the property. With that right comes the right to deny the owners the opportunity to live at the property, if doing so would interfere with his rights.
What can you do? That depends, first on more clarity to your situation as I and others have requested. Since your father is not living there, perhaps he is waiving his life estate. Perhaps your parent's subsequent divorce nullified this. You should seek out a local real estate attorney and bring to him or her any documents which show current ownership and whatever it is you think your parents set up.
To questioners from West Virginia & New York: Although I am licensed to practice in your state, I practice on a day-to-day basis in Massachusetts. I answer questions in your state in areas of the law in which I practice, and in which I feel comfortable trying to offer you assistance based on my knowledge of specific statutes in your state and/or general principles applicable in all states. It is always best, however, to work with attorneys and court personnel in your own area to deal with specific problems and factual situations.