Ok, continued investigation shows that actually it happens when the image size is too large to display on the screen. Provided I keep the display size less than the screen size, then everything seems really fast indeed. The moment I zoom in to a particular section of the image, so that the total size of the image is greater than the screen, I get everything slowing down. The performance falls through the floor. What is photoshop doing differently when the image is larger than the display?