Also the Dutch took tea to Europe as well in the early 1600s. So there are many entry points of tea to the west. The Dutch were all over Southeast Asia which also had Chinese tea as well.

Perhaps the Romans didn't like warm drinks (^_^) and stuck to wine. I'm sure they did see some Middle Eastern people drink tea though.