PhD and post-doc positions in Machine Learning and Natural Language Processing

Job description:

There are open positions for PhD students and postdoctoral researchers in the scope of the ERC project DECOLLAGE (“Deep Cognition Learning for Language Generation”).

Large-scale language models have led to impressive results in many NLP tasks, exhibiting transfer and few-shot learning capabilities. When interacting with such systems, users commonly find them capable of reasoning, planning, and explaining their decisions, often in convincing ways. However, despite the enormous advances in the last years, current deep learning models for NLP are still very limited in fundamental ways and many important ingredients are still missing to achieve a satisfactory level of “intelligence”. Some of these limitations partly stem from their monolithic architectures, which are good for some perceptual tasks, but unsuitable for tasks requiring higher-level cognition.

The overarching goal of DECOLLAGE is to attack these fundamental problems by bringing together tools and ideas from machine learning, sparse modeling, information theory, and cognitive science, in an interdisciplinary approach.

Three research directions are:
1. Designing new components for utility guidance, control, and contextualization. This will endow the model with the ability to predict its own quality (an “inner voice” or a critic) and to handle contextual information (e.g. document-level, conversation-level, meta-information about the surrounding environment), doing so in a modular, selective, and efficient manner.
2. Developing dynamic memory structures that facilitate continual learning, by supporting efficient reading and writing access, fast adaptation, and representation of world and self-knowledge. We will exploit synergies with sparse modeling and information retrieval.
3. Formalizing and implementing new mathematical models for sparse communication, bridging the gap between discrete (symbolic) and continuous representations, and developing techniques to integrate multiple modalities (such as text, speech, and image signals) into a shared representation space. This will draw links between information theory, formal languages, and neuroscience.

We will apply these innovations to highly challenging language generation tasks, including machine translation and open-ended generation.

See here for more information about the SARDINE Lab and some of our recent publications.

The specific roles assigned to the project members will take into account their research interests and their background.

The postdoc is a 2-year position, with the possibility of extension up to 3 years. The salary range is 2500-3000 EUR (free of tax), depending on the candidate’s experience. The PhD is a 4-year position, with salary range is 1500-2000 EUR (free of tax). These positions can potentially evolve into “ELLIS PhD/Post-Docs” and include a visit to an exchange institution in the ELLIS network.

If interested in this position, please send an email with your CV to andre.t.martins AT tecnico.ulisboa.pt. Feel free to contact me for any further information.

Working environment:

The host institution (Instituto de Telecomunicações) is a leading R&D institution located in the campus of Instituto Superior Técnico (IST) at the University of Lisbon, where I am an Associate Professor. IST is the top engineering school in Portugal, with 11,000+ students and 700+ faculty members, currently hosting 4 ERC grants.

The host institution is part of the Lisbon Unit for Machine Learning and Intelligent Systems (LUMLIS) and part of the ELLIS network. Collaborations with other ELLIS members are encouraged.

Part of the work may be done in collaboration with Unbabel, a scale-up translation company with unique assets (a crowd of 10,000+ human translators, and post-edited translations in 50+ language pairs), and with Champalimaud Centre for the Unknown, a cutting-edge research center in neuroscience.

Lisbon is one of the oldest cities in Western Europe, being the major political, economic and cultural centre of Portugal. It enjoys a Mediterranean climate, having one of the warmest winters of any metropolis in Europe and a long summer season lasting from June to September. Portugal has an outstanding coastline and is home to some of the most stunning beaches in Europe, being a popular spot for surfers and other water sport enthusiasts.

Lisbon is also home to the Lisbon Machine Learning School (LxMLS), which every year brings together 200+ students and the world leading researchers in machine learning and natural language processing.