Kinematic imaging of salticid eye tubes
with noisy X-ray video
The principal eyes of jumping spiders enable remarkable visual behaviors but key optical properties remain unknown. Specifically, we seek to measure changes in eye tube morphology in live spiders as they move in response to controlled stimuli. This required the development of new methodologies in both biology and video processing.
Video Capture
Using beam 32-ID of Argonne National Laboratory's Advanced Photon Source, the eye tubes and other internal structures of live, restrained spiders could be observed at video rates. Spiders were responsive for up to 10 minutes of stimulus exposure, which prompted smooth and repetitive eye motions in response to translating, rotating, and looming shapes presented on a 2D screen.
Video Processing
Our multi-filter video processing pipeline enhances spatiotemporal contrast in noisy X-ray videos. This reveals the internal position of moving salticid eye tubes. In the enhanced videos, eye tube features can be tracked with a combination of automatic and human-informed tools, enabling new analyses of spider eye kinematics with implications for their visual capacities.
Raw Video
raw.mp4Retina Tracking on Processed Video
2pit_kalman_spline_fixed.mp4This research was supported by NSF IOS-1831767 and used resources of the Advanced Photon Source, a U.S. Department of Energy (DOE) Office of Science User Facility operated for the DOE Office of Science by Argonne National Laboratory under Contract No. DE-AC02-06CH11357. This work was presented at Sensorium 2023. Code is available here.