Stereopsis without correspondence

Stereopsis without correspondence by Read JCA, Read2022.pdf (1.9 MiB) - Stereopsis has traditionally been considered a complex visual ability, restricted
to large-brained animals. The discovery in the 1980s that insects, too, have
stereopsis, therefore, challenged theories of stereopsis. How can such simple
brains see in three dimensions? A likely answer is that insect stereopsis has
evolved to produce simple behaviour, such as orienting towards the closer
of two objects or triggering a strike when prey comes within range. Scientific
thinking about stereopsis has been unduly anthropomorphic, for example
assuming that stereopsis must require binocular fusion or a solution of the
stereo correspondence problem. In fact, useful behaviour can be produced
with very basic stereoscopic algorithms which make no attempt to achieve
fusion or correspondence, or to produce even a coarse map of depth across
the visual field. This may explain why some aspects of insect stereopsis
seem poorly designed from an engineering point of view: for example,
paying no attention to whether interocular contrast or velocities match. Such
algorithms demonstrably work well enough in practice for their species, and
may prove useful in particular autonomous applications.
This article is part of a discussion meeting issue ‘New approaches to 3D
vision’.

A computational model of stereoscopic prey capture in praying mantises

A computational model of stereoscopic prey capture in praying mantises by O’Keeffe J, Yap SH, Llamas-Cornejo I, Nityananda V, Read JCA, OKeeffeYapLlamasCornejoNityanandaRead2022.pdf (4.2 MiB) - We present a simple model which can account for the stereoscopic sensitivity of praying mantis predatory strikes. The model consists of a single “disparity sensor”: a binocular neuron sensitive to stereoscopic disparity and thus to distance from the animal. The model is based closely on the known behavioural and neurophysiological properties of mantis stereopsis. The monocular inputs to the neuron reflect temporal change and are insensitive to contrast sign, making the sensor insensitive to interocular correlation. The monocular receptive fields have a excitatory centre and inhibitory surround, making them tuned to size. The disparity sensor combines inputs from the two eyes linearly, applies a threshold and then an exponent output nonlinearity. The activity of the sensor represents the model mantis’s instantaneous probability of striking. We integrate this over the stimulus duration to obtain the expected number of strikes in response to moving targets with different stereoscopic disparity, size and vertical disparity. We optimised the parameters of the model so as to bring its predictions into agreement with our empirical data on mean strike rate as a function of stimulus size and disparity. The model proves capable of reproducing the relatively broad tuning to size and narrow tuning to stereoscopic disparity seen in mantis striking behaviour. Although the model has only a single centre-surround receptive field in each eye, it displays qualitatively the same interaction between size and disparity as we observed in real mantids: the preferred size increases as simulated prey distance increases beyond the preferred distance. We show that this occurs because of a stereoscopic “false match” between the leading edge of the stimulus in one eye and its trailing edge in the other; further work will be required to find whether such false matches occur in real mantises. Importantly, the model also displays realistic responses to stimuli with vertical disparity and to pairs of identical stimuli offering a “ghost match”, despite not being fitted to these data. This is the first image-computable model of insect stereopsis, and reproduces key features of both neurophysiology and striking behaviour.

Binocular responsiveness of projection neurons of the praying mantis optic lobe in the frontal visual field

Binocular responsiveness of projection neurons of the praying mantis optic lobe in the frontal visual field by Rosner R, Tarawneh G, Lukyanova V, Read JCA, Full-text-available-for-free-at-https.txt (94 B) - Praying mantids are the only insects proven to have stereoscopic vision (stereopsis): the ability to perceive depth from the slightly shifted images seen by the two eyes. Recently, the first neurons likely to be involved in mantis stereopsis were described and a speculative neuronal circuit suggested. Here we further investigate classes of neurons in the lobula complex of the praying mantis brain and their tuning to stereoscopically-defined depth. We used sharp electrode recordings with tracer injections to identify visual projection neurons with input in the optic lobe and output in the central brain. In order to measure binocular response fields of the cells the animals watched a vertical bar stimulus in a 3D insect cinema during recordings. We describe the binocular tuning of 19 neurons projecting from the lobula complex and the medulla to central brain areas. The majority of neurons (12/19) were binocular and had receptive fields for both eyes that overlapped in the frontal region. Thus, these neurons could be involved in mantis stereopsis. We also find that neurons preferring different contrast polarity (bright vs dark) tend to be segregated in the mantis lobula complex, reminiscent of the segregation for small targets and widefield motion in mantids and other insects.

Second-order cues to figure motion enable object detection during prey capture by praying mantises

Second-order cues to figure motion enable object detection during prey capture by praying mantises by Nityananda V, O’Keeffe J, Umeton D, Simmons A, Read JCA, NityanandaOKeeffeUmetonSimmonsRead2019.pdf (1.6 MiB) - Detecting motion is essential for animals to perform a wide variety of functions. In order to do so, animals could exploit motion cues, including both first-order cues—such as luminance correlation over time—and second-order cues, by correlating higher-order visual statistics. Since first-order motion cues are typically sufficient for motion detection, it is unclear why sensitivity to second-order motion has evolved in animals, including insects. Here, we investigate the role of second-order motion in prey capture by praying mantises. We show that prey detection uses second-order motion cues to detect figure motion. We further present a model of prey detection based on second-order motion sensitivity, resulting from a layer of position detectors feeding into a second layer of elementary-motion detectors. Mantis stereopsis, in contrast, does not require figure motion and is explained by a simpler model that uses only the first layer in both eyes. Second-order motion cues thus enable prey motion to be detected, even when perfectly matching the average background luminance and independent of the elementary motion of any parts of the prey. Subsequent to prey detection, processes such as stereopsis could work to determine the distance to the prey. We thus demonstrate how second-order motion mechanisms enable ecologically relevant behavior such as detecting camouflaged targets for other visual functions including stereopsis and target tracking.

Motion-in-depth perception and prey capture in the praying mantis Sphodromantis lineola

Motion-in-depth perception and prey capture in the praying mantis Sphodromantis lineola by Nityananda V, Joubier C, Tan J, Tarawneh G, Read JCA, NityanandaJoubierTanTarawnehRead2019.pdf (0.8 MiB) - Perceiving motion-in-depth is essential to detecting approaching
or receding objects, predators and prey. This can be achieved
using several cues, including binocular stereoscopic cues such
as changing disparity and interocular velocity differences, and
monocular cues such as looming. Although these have been
studied in detail in humans, only looming responses have been well
characterized in insects and we know nothing about the role of
stereoscopic cues and how they might interact with looming cues. We
used our 3D insect cinema in a series of experiments to investigate
the role of the stereoscopic cues mentioned above, as well as
looming, in the perception of motion-in-depth during predatory strikes
by the praying mantis Sphodromantis lineola. Our results show that
motion-in-depth does increase the probability of mantis strikes but
only for the classic looming stimulus, an expanding luminance edge.
Approach indicated by radial motion of a texture or expansion of a
motion-defined edge, or by stereoscopic cues, all failed to elicit
increased striking. We conclude that mantises use stereopsis to
detect depth but not motion-in-depth, which is detected via looming.

Pattern and Speed Interact to Hide Moving Prey

Pattern and Speed Interact to Hide Moving Prey by Umeton D, Tarawneh G, Fezza E, Read JCA, Rowe C, UmetonTarawnehFezzaReadRowe2019.pdf (1.1 MiB) - Evolutionary biologists have long been fascinated by camouflage patterns that help animals reduce their chances of being detected by predators. However, patterns that hide prey when they remain stationary, such as those that match their backgrounds, are rendered ineffective once prey are moving. The question remains: can a moving animal ever be patterned in a way that helps reduce detection by predators? One long-standing idea is that high-contrast patterns with repeated elements, such as stripes, which are highly visible when prey are stationary, can actually conceal prey when they move fast enough [11, 12, 13, 14]. This is predicted by the “flicker fusion effect,” which occurs when prey move with sufficient speed that their pattern appears to blur, making them appear more featureless and become less conspicuous against the background [2, 8]. However, although this idea suggests a way to camouflage moving prey, it has not been empirically tested, and it is not clear that it would work at speeds that are biologically relevant to a predator [13]. Combining psychophysics and behavioral approaches, we show that speed and pattern interact to determine the detectability of prey to the praying mantis (Sphodromantis lineola) and, crucially, that prey with high-contrast stripes become less visible than prey with background-matching patterns when moving with sufficient speed. We show that stripes can reduce the detection of moving prey by exploiting the spatiotemporal limitations of predator perception, and that the camouflaging effect of a pattern depends upon the speed of prey movement.

A neuronal correlate of insect stereopsis

A neuronal correlate of insect stereopsis by Rosner R, von Hadeln J, Tarawneh G, Read JCA, Rosner_et_al-2019-Nature_Communications.pdf (2.9 MiB) - A puzzle for neuroscience-and robotics-is how insects achieve surprisingly complex behaviours with such tiny brains. One example is depth perception via binocular stereopsis in the praying mantis, a predatory insect. Praying mantids use stereopsis, the computation of distances from disparities between the two retinal images, to trigger a raptorial strike of their forelegs when prey is within reach. The neuronal basis of this ability is entirely unknown. Here we show the first evidence that individual neurons in the praying mantis brain are tuned to specific disparities and eccentricities, and thus locations in 3D-space. Like disparity-tuned cortical cells in vertebrates, the responses of these mantis neurons are consistent with linear summation of binocular inputs followed by an output nonlinearity. Our study not only proves the existence of disparity sensitive neurons in an insect brain, it also reveals feedback connections hitherto undiscovered in any animal species.

Apparent Motion Perception in the Praying Mantis: Psychophysics and Modelling

Apparent Motion Perception in the Praying Mantis: Psychophysics and Modelling by Tarawneh G, Jones L, Nityananda V, Rosner R, Rind C, Read JCA, TarawnehNityanandaJonesNityanandaRosnerRindRead2018.pdf (0.9 MiB) - Apparent motion is the perception of motion created by rapidly presenting still frames in which objects are displaced in space. Observers can reliably discriminate the direction of apparent motion when inter-frame object displacement is below a certain limit, Dmax. Earlier studies of motion perception in humans found that Dmax is lower-bounded at around 15 arcmin, and thereafter scales with the size of the spatial elements in the images. Here, we run corresponding experiments in the praying mantis Sphodromantis lineola to investigate how Dmax scales with the element size. We use random moving chequerboard patterns of varying element and displacement step sizes to elicit the optomotor response, a postural stabilization mechanism that causes mantids to lean in the direction of large-field motion. Subsequently, we calculate Dmax as the displacement step size corresponding to a 50% probability of detecting an optomotor response in the same direction as the stimulus. Our main findings are that the mantis Dmax scales roughly as a square-root of element size and that, in contrast to humans, it is not lower-bounded. We present two models to explain these observations: a simple high-level model based on motion energy in the Fourier domain and a more-detailed one based on the Reichardt Detector. The models present complementary intuitive and physiologically-realistic accounts of how Dmax scales with the element size in insects. We conclude that insect motion perception is limited by only a single stage of spatial filtering, reflecting the optics of the compound eye. In contrast, human motion perception reflects a second stage of spatial filtering, at coarser scales than imposed by human optics, likely corresponding to the magnocellular pathway. After this spatial filtering, mantis and human motion perception and Dmax are qualitatively very similar.

Contrast thresholds reveal different visual masking functions in humans and praying mantises

Contrast thresholds reveal different visual masking functions in humans and praying mantises by Tarawneh G, Nityananda V, Rosner R, Errington S, Herbert W, Arranz-Paraíso S, Busby N, Tampin J, Read JCA, Serrano-Pedraza I, TarawnehNityanandaRosnerErringtonHerbertArranzParaisoBusbyTampinSerranoSerranoPedraza2018.pdf (2.3 MiB) - Recently, we showed a novel property of the Hassenstein–Reichardt detector, namely that insect motion detection can be masked by ‘undetectable’ noise, i.e. visual noise presented at spatial frequencies at which coherently moving gratings do not elicit a response (Tarawneh et al., 2017). That study compared the responses of human and insect motion detectors using different ways of quantifying masking (contrast threshold in humans and masking tuning function in insects). In addition, some adjustments in experimental procedure, such as presenting the stimulus at a short viewing distance, were necessary to elicit a response in insects. These differences offer alternative explanations for the observed difference between human and insect responses to visual motion noise. Here, we report the results of new masking experiments in which we test whether differences in experimental paradigm and stimulus presentation between humans and insects can account for the undetectable noise effect reported earlier. We obtained contrast thresholds at two signal and two noise frequencies in both humans and praying mantises (Sphodromantis lineola), and compared contrast threshold differences when noise has the same versus different spatial frequency as the signal. Furthermore, we investigated whether differences in viewing geometry had any qualitative impact on the results. Consistent with our earlier finding, differences in contrast threshold show that visual noise masks much more effectively when presented at signal spatial frequency in humans (compared to a lower or higher spatial frequency), while in insects, noise is roughly equivalently effective when presented at either the signal spatial frequency or lower (compared to a higher spatial frequency). The characteristic difference between human and insect responses was unaffected by correcting for the stimulus distortion caused by short viewing distances in insects. These findings constitute stronger evidence that the undetectable noise effect reported earlier is a genuine difference between human and insect motion processing, and not an artefact caused by differences in experimental paradigms.

A Novel Form of Stereo Vision in the Praying Mantis

A Novel Form of Stereo Vision in the Praying Mantis by Nityananda V, Tarawneh G, Henriksen S, Umeton D, Simmons A, Read JCA , NityanandaTarawnehHenriksenUmetonSimmonsRead2018.pdf (2.5 MiB) - Stereopsis is the ability to estimate distance based on the different views seen in the two eyes. It is an important model perceptual system in neuroscience and a major area of machine vision. Mammalian, avian, and almost all machine stereo algorithms look for similarities between the luminance-defined images in the two eyes, using a series of computations to produce a map showing how depth varies across the scene. Stereopsis has also evolved in at least one invertebrate, the praying mantis. Mantis stereopsis is presumed to be simpler than vertebrates’, but little is currently known about the underlying computations. Here, we show that mantis stereopsis uses a fundamentally different computational algorithm from vertebrate stereopsis --
rather than comparing luminance in the two eyes’ images directly, mantis stereopsis looks for regions of the images where luminance is changing. Thus, while there is no evidence that mantis stereopsis works at all with static images, it successfully reveals the distance to a moving target even in complex visual scenes with targets that are perfectly camouflaged against the background in terms of texture. Strikingly, these insects outperform human observers at judging stereoscopic distance when the pattern of luminance in the two eyes does not match. Insect stereopsis has thus evolved to be computationally efficient while being robust to poor image resolution and to discrepancies in the pattern of luminance between the two eyes.