OK - a few things here.
First, to get a student visa, you need a form called an I-20 issued by the school where you will attend. You need to decide where you want to go to school and speak - the earlier the better, with the foreign student adviser (often called a "DSO") at that school.
For most people who have been here without legal status for a long time, leaving the U.s. is a problem: 180 days of "unlawful presence time" means a three-year bar to coming back, and a year or more of "unlawful presence time" means a ten-year bar to coming back.
BUT: even if out of status, a person under 18 years old doesn't build up "unlawful presence time." So, as long as you leave the U.S. before 180 days pass from your 18th birthday, these reentry bars aren't a problem for you.
HOWEVER, here's what WILL be a problem: to get an F-1 student visa, you needs to prove "non-immigrant intent" - the intent to return to your home abroad at the end of your time here as a student. The U.S. consulate or embassy where you apply in Mexico will look to your ties to Mexico vs the U.S. in determining this, and here's where there's a problem: you've been living here a long time, and likely have family here, but have few ties to Mexico.
Speak to the school DSO and perhaps an attorney concerning the chances here. Good luck!!
The time that you have remained in the US out of status will impact your future ability to obtain a student visa. You should consult with an immigration attorney before leaving the US to determine if obtaining a visa in Mexico is a possibility.
Normally, a student must be accepted to a school that accepts foreign students. The school will issue an I-20 form that must be submitted to USCIS or the US Embassy. However, your prior status must be considered before you leave the country.
In order to get a student visa, you have to prove to the U.S. consular officer in Mexico that you have "nonimmigrant" intent. That means you have to convince them that you will return to Mexico at the end of your program of study. Because you have been in the U.S. for so long, they will presume that when you return, you intend to stay here instead of going back to Mexico, and they will probably deny the visa. Speak to a lawyer before you consider leaving the U.S.
(You would have been one of those that could be helped by the DREAM Act but that was never passed into law.)