Overview

Any action, for example picking up a mug, requires the cooperation of several systems in the brain, each of which has a different role in the execution of the action. When thirst or habit triggers an intention to find something to drink the first job is likely to be to locate a cup, mug or glass, and finding it is the function of the gaze control system. Objects are nearly always fixated by the foveas of the eyes before any action occurs, and this may involve movements not just of the eyes, but of the head and body as well.

If the mug is visible in peripheral vision it may be targeted directly, but if it is initially out of sight the gaze system will need to access remembered information about where the mug is likely to be before it can be located. Once the mug has been imaged on the fovea, the motor system can set about the task of grasping it and lifting it. To do that it requires information supplied by the image processing regions of the visual system. Vision has first to confirm that the object is indeed a mug, and then determine the direction and distance the arm and hand must travel. For the efficient performance of the grasp the orientation of the handle must be worked out, so that the hand can be shaped to it during the extension of the arm. Thus the whole task requires the gaze system to locate objects, the motor system to operate on them, and the visual system to supply the information that makes this possible. Although the three systems just described may seem adequate for the simple action of grasping a mug it is clear that there has to be fourth system that selects what task to perform. Suppose our subject, having had his cup of tea, decides to read a book. The gaze system needs to locate the bookshelf and the books, the visual system has to be primed to recognise the right title, and then to provide the motor system with information to guide the hands in extracting the book from the row. Although the overall locating, acting and information-supplying functions are the same as they were in dealing with the mug, the details are very different, and they depend crucially on the task. Each system has to be primed with information from some supervisory body that knows the task and its requirements.



In our lab we are interested in all aspects of this active vision architecture. For more details of our current active projects use the links above.