Apparently I lied by accident?

A conversation came up that apparently we'd had before and when I answered 4, this person accused me of lying and said that before, I'd answered 1. How does this work if I have absolutely no memory of the conversation or the initial question? And now I'm trying to remember: did I really lie; or did I just throw a random number into the open because I wasn't paying attention? I don't know. That's weird though. How could I lie and not remember doing it?