In some states the right to petition the court to modify custody orders is limited in a way that discourages repeated, frivolous, or trivial changes or actions that are more about harassing the other parent than meeting the best interests of the child. The concerns you describe do sound like matters that merit the attention of the court. You probably will find the assistance of an experienced family law attorney helpful in navigating the legal system and more likely to produce the result you want.
Best wishes for a favorable outcome, and please remember to designate a best answer.
This answer is offered as a public service for general information only and may not be relied upon as legal advice.
Attorney Sinclair did provide a good answer, and as Family Law attorney in Michigan I can tell you our state requires "a proper cause or s substantial change in circumstances to warrant a review of an existing child custody order". However, you may not need a change in custody, it sounds like you just need a change in parenting time. Your daughters father can still be an active part of her life but the week-on week-off schedule may not be best for your child. I suspect that due to your child age (you mentioned she was just taken off formula) he could have frequent but shorter visits.
Sign up to receive a 5-part series of useful information and advice about child custody law.