Doctoral student in Multimodal AI for Human-Robot Interaction

Project description

Third-cycle subject: Computer Science

We are looking for a motivated PhD student for the WASP WARA Media and Language project "BELLA: Building Expressive Language for Likeable Agents" at KTH Royal Institute of Technology's Department of Intelligent Systems, Division of Speech, Music, and Hearing (TMH).

This project aims to enhance Human-Robot Interaction (HRI) through embodied AI by integrating multimodal social cues and task-related actions into foundation models, enabling more natural and human-like communication. While Large Language Models have advanced AI significantly, current systems lack understanding and generation of crucial social cues like facial expressions, gestures, and gaze direction, as well as the ability to incorporate task-specific behaviors. This project aims to bridge these gaps.

Key Objectives:

1. Integrate multimodal perception: Develop methods to fuse foundation models with embodied and task-specific tags.
2. Generate multimodal outputs: Train models to produce relevant verbal responses, non-verbal behaviors, and task-related actions.
3. Evaluate comprehensively: Develop novel metrics to assess integrated models in HRI scenarios.

The Swedish AI-program WASP funds this project. WASP's graduate school fosters a strong multi-disciplinary, international network among PhD students, researchers, and industry through research visits, partner universities, and visiting lecturers.

Supervision: Researcher, André Pereira

What we offer

• The possibility to study in a dynamic and international research environment in collaboration with industries and prominent universities from all over the world.
• A workplace with many employee benefits and monthly salary according to KTH’s Doctoral student salary agreement.
• A postgraduate education at an institution that is active and supportive in matters pertaining to working conditions, gender equality and diversity as well as study environment.
• Help to relocate and be settled in Sweden and at KTH.

Admission requirements

To be admitted to postgraduate education (Chapter 7, 39 § Swedish Higher Education Ordinance), the applicant must have basic eligibility in accordance with either of the following:

• passed a second cycle degree (for example a master's degree), or
• completed course requirements of at least 240 higher education credits, of which at least 60 second-cycle higher education credits, or
• acquired, in some other way within or outside the country, substantially equivalent knowledge

In addition to the above, there is also a mandatory requirement for English equivalent to English B/6.

Selection

In order to succeed as a doctoral student at KTH you need to be goal oriented and persevering in your work. During the selection process, candidates will be assessed upon their ability to:

• independently pursue his or her work
• collaborate with others,
• have a professional approach and
• analyse and work with complex issues.

The ideal candidate has a strong background in AI, machine learning, or a related field, with experience in natural language processing (NLP) or multimodal machine learning (MML) preferred. Passion for social robotics is considered a plus.

After the qualification requirements, great emphasis will be placed on personal competency. 

Target degree: Doctoral degree

Information regarding admission and employment

Only those admitted to postgraduate education may be employed as a doctoral student. The total length of employment may not be longer than what corresponds to full-time doctoral education in four years ' time. An employed doctoral student can, to a limited extent (maximum 20%), perform certain tasks within their role, e.g. training and administration. A new position as a doctoral student is for a maximum of one year, and then the employment may be renewed for a maximum of two years at a time. In the case of studies that are to be completed with a licentiate degree, the total period of employment may not be longer than what corresponds to full-time doctoral education for two years.

Union representatives

Contact information for union representatives.

Doctoral section (Students’ union on KTH Royal Institute of Technology)

Contact information for doctoral section.

To apply for the position

Apply for the position and admission through KTH's recruitment system. It is the applicant’s responsibility to ensure that the application is complete in accordance with the instructions in the advertisement.

Applications must be received at the last closing date at midnight, CET/CEST (Central European Time/Central European Summer Time).

Applications must include:

• CV including your relevant professional experience and knowledge.
• Copies of diplomas and grades from previous university studies and certificates of fulfilled language requirements (see above). Translations into English or Swedish if the original document is not issued in one of these languages. Copies of originals must be certified.
• Representative publications or technical reports. For longer documents, please provide a summary (abstract) and a web link to the full text.

Other information

Striving towards gender equality, diversity and equal conditions is both a question of quality for KTH and a given part of our values.

For information about processing of personal data in the recruitment process.

According to The Protective Security Act (2018-585), the candidate must undergo and pass security vetting if the position is placed in a security class. Information regarding whether the position is subject to such a classification will be provided during the recruitment process.

We firmly decline all contact with staffing, recruitment agencies and job ad salespersons.

Disclaimer: In case of discrepancy between the Swedish original and the English translation of the job announcement, the Swedish version takes precedence.