The remoteness of your server should have nothing to do with the legality of your actions. There is a very serious dispute over whether you really are entitled to keep a copy of your CD music on your computer. But whether or not "your computer" relies on data available only to you but stored somewhere else, or is limited to your desktop only, should not affect how that dispute is resolved. This does not mean that one side or the other will not try to argue that it should affect it, but rather that in my view, given my understanding of these issues, it should not.
I am sorry if this is not too helpful. Unfortunately, this is the state of copyright law today! To get a sense of how complicated this is, see the link below.