It sounds like you have a temporary custody order that the judge is going to review at some time in the near future.
First, stay on your best behavior. No arguments/fights in front of the children, or anywhere else for that matter. No comments about the mother and her boyfriend, etc.
Second, get a copy of Va. Code 20-124.3. This is the "best interests" factors that the judge must and will consider in making visitation and custody determinations. Learn them. Live them.
Third, diligently and punctually exercise all access/visitation you are presently authorized to have. Arrive 5 minutes early for pick up and drop off. No drama exchanging the children. You are not going to get more time from the judge if you have not used the time already allowed. You may lose time if you can't begin or end visitation in a timely fashion.
Fourth, ask the mother for more time. Offer to do the transportation. Talk to your children's teachers or daycare providers. Ask if there are any doctor appointments you can attend. Again, be civil. Take "no" for any answer. It is a win/win for you and a lose/lose for her. If she denies you the opportunity to attend a doctor appointment, she loses on factor 7, and you made a reasonable request. If you get to attend the appointment, then you have demonstrated good parenting and can talk to the judge about factors 1, 3, 5 and 7.
Finally, prepare in advance and organize your evidence and argument for the judge using the best interest factors. Use the 10 factors as your outline.
The court can consider it to be a negative factor that the mother is not facilitating contact between you and the child.
The ideal situation, however, would be if you could learn to work together. Sometimes, mediation is helpful in working out a visitation schedule and in helping the parties communicates. You may want to try this. It is available at no cost through the court.
Sign up to receive a 5-part series of useful information and advice about child custody law.