I agree with you. Moreover, the eye has a fantastic ability to filter out differences in illumination. If you could design an experiment where someone who had been illuminated by different color temperature lights could be asked what color temperature the light was without remembering what it was from when they were first exposed to it, I bet that people would have difficulty doing so.