If the father only has supervised visits, why does he need to provide a letter so you can see family in Canada? Look at your decree and possession order. Determine if there are any grounds for the father to be required to grant you permission. If he is required to grant permission and fails to, you may need to seek court intervention. Basically, his failure to grant permission, during a period of time that is not his time of possession, interferes with your possession of the child.
I understand your frustration, believe me. I also understand the rules regarding having the other parent give consent to take the children across border. Unfortunately, you run severe risks by not obtaing the other parent's permission regardless of the realities of the day-to-day decision making of the custodial parent. Perhaps a letter from your spouse's attroeny to the children's father may be able to resolve the issue. Absent that, the children's father is likely unreasonably interfering, especially if he has allowed prior cross border visits, and this is not during his period of possession, and that is something that may require court intervention to resolve. I realize there can be petty disagreements between divorced parents that ultimately effect the children in a less than positive way.
On another note, Texas does allow sole conservatorship, but in extreme cases. The presumption is it is best to keep both parents involved in the child's life. This would not be a case to modify the conservatorship from "joint managing" to "sole" based on this example.
Good luck. If you think I can help you, please let me know.