Just to be clear, this was asked at the drive up window, not during an "extended" immigration check.
Not at the window. Inside.
Border guards at the window may ask you to go inside for a more extensive background check. I'm sure they would do this if you are suspicious, but they also randomly select people. I have been through this check twice in the many years of crossing the Canadian border.
They will take you passports and run some checks through their system. After we came out "clean" is when they shoot out the "trick question". It comes across as a, "Oh, by the way" type question as they are handing you back your passports so it could catch some people off guard. They make sound as if you're all clear no matter what you answer by the way they deliver the question. Both times I was brought inside the office the question was asked and asked in the same manner.