Since you were never found to be a refugee, and based on the facts you present, it does not seem that traveling back wold pose you any problems.
Samuel Ouya Maina, Esq. 415.391.6612 email@example.com Law Offices of S. Ouya Maina, PC 332 Pine Street, Suite 707 San Francisco, CA 94104
Given that you did not obtain a green card through asylum, traveling back to your country of birth/nationality will not have any adverse effects on your green card status.
Behar Intl. Counsel 619.234.5962 Kindly be advised that the answer above is only general in nature cannot be construed as legal advice, given that not enough facts are known. It is your responsibility to retain a lawyer to analyze the facts specific to your particular situation in order to give you specific advice. Specific answers will require cognizance of all pertinent facts about your case. Any answers offered on Avvo are of a general nature only, and are not meant to create an attorney-client relationship.
As you appeared to be aware for persons filing for asylum or having been granted status in United States based on asylum, travel to the home country may be an issue. However, since you obtain your resident status based on a visa petition from your spouse travel to your own country should not be a concern for your immigration status.