I've long been intrigued by the 1995 Nature paper by Julie Harris and Andrew Parker, where they show that people perform better on a disparity task when the stimulus uses white and black dots on a grey background, than when the dots are all white or all black. They explain the effect by arguing that the stereo correspondence problem - that is, matching up which features in the two eyes correspond to the same object in space - is easier with black and white dots. Mixed colours instantly halves the complexity of the problem, because you know that a black dot can't match with a white dot. That does sound very reasonable, but my problem is that I come at stereo correspondence from the perspective of the energy model, which effectively implements cross-correlation between the two eyes' images. Cross-correlation doesn't "see" dots at all. I couldn't figure out how to implement Julie and Andrew's explanation using current models of cells in early visual cortex.
I did, however, think I could see a way round the problem. Julie and Andrew's paper started with an easy stereo task - seeing which of two adjacent planes was closer. The planes were defined by dots which were randomly scattered in X and Y, but all at the same Z (where the Z axis defines distance from the observer). They then made the task harder by introducing disparity noise, i.e. giving each dot some jitter in Z. This meant you had to average over several dots in order to get a good estimate of mean Z.
When I coded up this stimulus and had a look at it, it immediately struck me that it didn't noticeably challenge stereo correspondence. I felt I could clearly see each dot in space, indicating that my brain had successfully solved the correspondence problem. But the task was hard, because it wasn't obvious which cloud of dots was closer. Once I'd realised that, I thought I might have a way out of my difficulty. Stereo correspondence would proceed by, essentially, cross-correlation, and mixed black/white dots would offer no advantage over all-white and all-black dots. But then, in order to do Julie and Andrew's task, a higher brain area would have to figure out which dots to average over. Maybe that brain area is only able to average over a certain number of dots within each category, and therefore adding a different category ("black" as well as "white") improves performance. So when Xavier Vaz, a Biomedical Sciences undergraduate, did his project in my lab, I had him test this theory by comparing Julie and Andrew's original task with a different one designed to challenge stereo correspondence, but to be trivial once correspondence had been achieved. I was confident the mixed-colour advantage would show up on the original task and be abolished in our new experiment.
And I was completely wrong. Performance on both tasks was clearly better for mixed black-and-white dots. Julie and Andrew's result holds not only in their original task, but also in this new version of it. And I still don't have a clue how to reconcile this with my understanding of how disparity is encoded in early visual cortex.
I did, however, think I could see a way round the problem. Julie and Andrew's paper started with an easy stereo task - seeing which of two adjacent planes was closer. The planes were defined by dots which were randomly scattered in X and Y, but all at the same Z (where the Z axis defines distance from the observer). They then made the task harder by introducing disparity noise, i.e. giving each dot some jitter in Z. This meant you had to average over several dots in order to get a good estimate of mean Z.
When I coded up this stimulus and had a look at it, it immediately struck me that it didn't noticeably challenge stereo correspondence. I felt I could clearly see each dot in space, indicating that my brain had successfully solved the correspondence problem. But the task was hard, because it wasn't obvious which cloud of dots was closer. Once I'd realised that, I thought I might have a way out of my difficulty. Stereo correspondence would proceed by, essentially, cross-correlation, and mixed black/white dots would offer no advantage over all-white and all-black dots. But then, in order to do Julie and Andrew's task, a higher brain area would have to figure out which dots to average over. Maybe that brain area is only able to average over a certain number of dots within each category, and therefore adding a different category ("black" as well as "white") improves performance. So when Xavier Vaz, a Biomedical Sciences undergraduate, did his project in my lab, I had him test this theory by comparing Julie and Andrew's original task with a different one designed to challenge stereo correspondence, but to be trivial once correspondence had been achieved. I was confident the mixed-colour advantage would show up on the original task and be abolished in our new experiment.
And I was completely wrong. Performance on both tasks was clearly better for mixed black-and-white dots. Julie and Andrew's result holds not only in their original task, but also in this new version of it. And I still don't have a clue how to reconcile this with my understanding of how disparity is encoded in early visual cortex.