A new method of multimodal multi-level fusion integrating contextual information obtained from spoken input and visual scene analysis is being developed at Université Catholique de Louvain in the frame of the EU-funded SIMILAR Network of Excellence. An experimented application example is the “intelligent diary” aimed at proactively assisting elderly people living alone at home to perform their daily activities, to prolong their safe and secure personal autonomy, to support their active ageing and social cohesion.