To be eligible for naturalization, you must meet the criteria outlined in the Immigration and Nationality Act along with the corresponding Code of Federal Regulations. The initial inquiry for eligibility is how long you have been a permanent resident. Generally, you must have been a permanent resident for five years. There are some exceptions; if you are married to a U.S. citizen or you received you permanent resident status through a VAWA petition. There is no exception for green card holders who obtained their status through asylum.
The date your clock begins to tick is the date your I-485 was approved. This is generally reflected on the approval notice (Form I-797) and your green card (where it states "Resident Since").
Contact an immigration attorney to determine your other eligibility factors.