Kenny Gamble explained to National Public Radio in 2008 that he and Huff got the idea for the song from trips to a little bar downstairs in the Schubert Building, which was where their record company was located. Said Gamble: “This guy used to come into the bar every day – little guy that looked like a judge. We’re songwriters, so we’re always thinking about a song. The next day he came in again, and every day after he’d come in, this girl would come in 10-15 minutes after he’d get there, and they’d sit in the same booth, then go to the jukebox and play the same songs. We said, ‘That’s me and Mrs. Jones.’ Then, when they’d get ready to leave, he would go his way and she would go hers. It could have been his daughter, his niece, anybody, but we created a story that there was some kind of romantic connection between these people, so we went upstairs to our office and wrote the song.”