Où Est Votre Bébé? AKA Dude Where's Your Baby?

A couple of weeks ago, while abroad, I met up with an old friend and her baby.  Given that we were in a nice city, instead of catching up at home, we took the city by storm and spent our days packed full of site-seeing and shopping.  At one point, my friend took her son into the restroom and I kept walking around with the stroller, which may or may not have been laden down with my shopping bags.  As I meandered around the site, another tourist stopped me and said,

"Pardon, où est votre bébé?"

"Umm, with his mother, my friend, in the bathroom."

Did you think I was stealing a stroller or that I forgot my child?  I'm not sure which is worse.