If you were not required to file tax returns because you did not earn enough income to reach the minimum, then your naturalization application will not suffer because you didn't file. The key question is whether you ever filed an income tax return as a nonresident, which would be a problem. Also whether you ever failed to file a tax return when you were required to file.
If you did not have to file a tax return, then not doing so will not be held against you. As long as you did file a return each year in which you were required to file you should be OK.
This answer does not create an attorney-client relationship and is based only on the limited information provided in the question.
Your application will not be adversely effected if you were not required to file income tax returns. If, however, you worked and just decided not to file income tax returns, then you will have an issue. Additionally, if you ever filed income tax returns as a "non resident" that time period that you considered yourself to be a non-resident will not be counted towards the time needed for naturalization.
DISCLAIMER: This answer is intended to provide general information only. No attorney-client relationship has been created by this posting.