Doctoral students in generative AI for 3D gesture motion

Project description

Third-cycle subject: Computer science

In face-to-face communication, humans use co-speech gestures – such as hand, arm, head, and body movements – to enhance clarity, naturalness, and efficiency. Our lab is a pioneer and global leader in leveraging 3D motion-capture data and generative AI to synthesise these gestures for virtual characters and humanoid robots.

The specific focus of this project is to use data on human opinion – e.g., subjective ratings of whether a gesture looks good visually, or whether it’s a good fit for the character’s speech – to further improve gesture generation. You will not only try to predict human opinion of these aspects, but also use ratings data to further improve models using direct preference optimisation (DPO) or other reinforcement learning from human feedback (RLHF) techniques. There may also be opportunities to pursue improved benchmarking and evaluation practices for gesture motion, sign-language synthesis, or joint multimodal synthesis of speech audio and gesture motion together.

Supervision: Assistant Professor Gustav Eje Henter; co-supervisors Prof. Jonas Beskow and Assistant Prof. Éva Székely

What we offer

• The possibility to study in a dynamic and international research environment in collaboration with industries and prominent universities from all over the world.
• A workplace with many employee benefits and monthly salary according to KTH’s Doctoral student salary agreement.
• A postgraduate education at an institution that is active and supportive in matters pertaining to working conditions, gender equality and diversity as well as study environment.
• Help to relocate and be settled in Sweden and at KTH.
• Opportunity to attend at least one international research conference per year.
• Expert education in a field in great demand nationally and internationally.

Admission requirements

To be admitted to postgraduate education (Chapter 7, 39 § Swedish Higher Education Ordinance), the applicant must have basic eligibility in accordance with either of the following:

• passed a second cycle degree (for example a master's degree), or
• completed course requirements of at least 240 higher education credits, of which at least 60 second-cycle higher education credits, or
• acquired, in some other way within or outside the country, substantially equivalent knowledge

In addition to the above, there is also a mandatory requirement for English equivalent to English B/6.

Selection

In order to succeed as a doctoral student at KTH you need to be goal oriented and persevering in your work. During the selection process, candidates will be assessed upon their ability to:

• independently pursue his or her work
• collaborate with others,
• have a professional approach and
• analyze and work with complex issues.

Other key traits are curiosity, attention to detail, and a strong ability to clearly communicate scientific ideas in speech and writing.

The applicant needs to demonstrate excellent command of deep learning, including strong programming skills in PyTorch or similar. Publications or other experience in areas like motion capture, 3D animation, speech technology, probability, and generative AI methods are a plus, as is an openness toward interdisciplinary research.

After the qualification requirements, great emphasis will be placed on personal skills. 

Target degree: Doctoral degree

Information regarding admission and employment

Only those admitted to postgraduate education may be employed as a doctoral student. The total length of employment may not be longer than what corresponds to full-time doctoral education in four years' time. An employed doctoral student can, to a limited extent (maximum 20%), perform certain tasks within their role, e.g. training and administration. A new position as a doctoral student is for a maximum of one year, and then the employment may be renewed for a maximum of two years at a time. In the case of studies that are to be completed with a licentiate degree, the total period of employment may not be longer than what corresponds to full-time doctoral education for two years.

Union representatives

Contact information for union representatives.

Doctoral section (Students’ union on KTH Royal Institute of Technology)

Contact information for doctoral section.

To apply for the position

Apply for the position and admission through KTH's recruitment system. It is the applicant’s responsibility to ensure that the application is complete in accordance with the instructions in the advertisement.

Applications must be received at the last closing date at midnight, CET/CEST (Central European Time/Central European Summer Time).

Applications must include the following elements:

• CV including your relevant professional experience and knowledge.
• Application letter with a brief description of why you want to pursue research studies, about what your academic interests are and how they relate to your previous studies and future goals. (Maximum 2 pages long)
• Copies of diplomas and grades from previous university studies and certificates of fulfilled language requirements (see above). Translations into English or Swedish if the original document is not issued in one of these languages. Copies of originals must be certified.
• Representative publications or technical reports. For longer documents, please provide a summary (abstract) and a web link to the full text.

Other information

Striving towards gender equality, diversity and equal conditions is both a question of quality for KTH and a given part of our values.

For information about processing of personal data in the recruitment process.

It may be the case that a position at KTH is classified as a security-sensitive role in accordance with the Protective Security Act (2018:585). If this applies to the specific position, a security clearance will be conducted for the applicant in accordance with the same law with the applicant's consent. In such cases, a prerequisite for employment is that the applicant is approved following the security clearance.

We firmly decline all contact with staffing and recruitment agencies and job ad salespersons.

Disclaimer: In case of discrepancy between the Swedish original and the English translation of the job announcement, the Swedish version takes precedence.