When you look at a good portion of arguments between a couple, some of them may come down to something like this:
In some way or another, he says something but does nothing with his actions to confirm or line up with his words. At some point, she might call him out on not loving her well with his actions in that moment. He says in frustration, "I never SAID that!" She answers in deeper frustration, "You don't HAVE to."
The implication is to say, "You don't have to say anything; your actions (or lack thereof) speak loud and clear as to what you think, feel, or intend toward me in this moment."
As Christians we tend to talk a lot about love. We say that we are prepared to love others. We say we love everyone, no matter who they are or what they do or how they live. At some point, different groups of people look at the Church and its Christians and say, "You don't love me!"
The Church and its Christians respond in frustration, "I never said I didn't love you."
People respond in deeper frustration, "You didn't have to," implying "your actions (or lack thereof) speak loud and clear as to what you think, feel, or intend toward me in this moment."