I like UChessMaster's explanation best. All the things Kisame said last time give Kakashi the info he needed to come to the conclusion he did.

I actually thought after he said it that Itachi WAS just kinding of staring impassively. I started to wonder if he wasn't actually looking at anyone before and they were too far away to see.