You need to connect to fetch the notes, so yeah, you need to not be vulnerable to a relay giving you a payload that would compromise your client.

That said, you would still need to not be vulnerable to a malicious payload even if the user manually enters the relay URL