He had to have left before Urahara.

Important points to note:

-Rukia knows urahara
-If Ishin was there at the same time as Urahara's banishment he would look familiar to Rukia while she was living there.
-Urahara was banished within Rukia's time as a seated member of the Gotei 13
-Yourichi left before Rukia which explains why she does not really know her

Not sure if these are all true I read the manga last year and have been doing the weekly thing for awhile now.