The short answer is that, yes, you can insist that they move the fence back to the property line.
What you should do, on the other hand, is a matter for a long talk with an experienced property attorney.
The neighbors asked, and you agreed. There is no legal restriction on you now revoking your earlier agreement. But I don't know that there is anything you can do to force the neighbor to put up a new fence.
The better approach might be to come to an agreement with the neighbor to put in a new fence on the boundary line and you can split the cost. If not, then you could put your own fence up just inside the property line to hide what you don't want to look at.
As for the tree, if it is on their property, they have every right to put it in. Unless doing so violates some sort of local ordinance.
You may want to consult with a local attorney before doing anything.