Perception of the external world involves not only the appearance of the surface of objects and scenes, but also the reality of the world behind it (deep shitsukan). The brain is thought to generate realistic shitsukan by modeling the external world in combination with the knowledge and body state. In this project, we will use the homology between neural network models trained on large-scale data and the brain’s information representation to reveal the neural representation of the 3D world based on quantitative predictions. By combining the brain-machine translation approach with the latest computer graphics and computer vision technologies, we aim to reconstruct the internal model of the world in the brain.

Co-Investigator