Generative AI for Video Games
Primary supervisor
Additional supervisors
- Ke Chen
Contact admissions office
Other projects with the same supervisor
- Automatic Emotion Detection, Analysis and Recognition
- Biologically-Plausible Continual Learning
- Zero-Shot Learning and Applications
- Ensemble Strategies for Semi-Supervised, Unsupervised and Transfer Learning
- Multi-task Learning and Applications
- Explainable and Interpretable Machine Learning
- Machine Learning and Cognitive Modelling Applied to Video Games
- Automatic Activity Analysis, Detection and Recognition
- Music Generation and Information Processing via Deep Learning
- Deep Learning for Temporal Information Processing
- Contextualised Multimedia Information Retrieval via Representation Learning
Funding
- Competition Funded Project (Students Worldwide)
This research project is one of a number of projects at this institution. It is in competition for funding with one or more of these projects. Usually the project which receives the best applicant will be awarded the funding. Applications for this project are welcome from suitably qualified candidates worldwide. Funding may only be available to a limited set of nationalities and you should read the full department and project details for further information.
Project description
Generative AI is one of the most promising methods for achieving the long-standing goal of Non-Player Character (NPC) imitating particular playing behaviors which are represented as rules, rewards, or human demonstrations in modern video games. However, directly applying existing generative AI techniques to video games can be challenging. First, the generative AI targets at learning a distribution from offline data by iteratively minimizing a proxy measure, e.g., a loss function. This proxy measure, however, may not exactly describe the target and desired behavior (a.k.a value misalignment) for the NPC to imitate. The resulting model, though possibly optimal w.r.t. the proxy measure, can lead to unnatural or unhuman-like behaviors for NPC. Second, the modern video games are rich in different interaction modalities, various task settings, and diversified playing roles. The data inputs for a generative model could thus be multi-modal, multi-task and multi-embodiment, which poses a great challenge for Generative AI as most generative models can only capture the distribution from a single modality.
Research objectives include: (1) developing human-centric generative models, i.e., the generated distribution in Generative AI will strike a balance between the target distribution (i.e., optimality) and the distribution human might incur, (i.e., naturalness/human-likeness) [1]. We propose the ad-hoc teaming between generative AI and the domain experts which consists of two steps: first, the generative AI learns a model from offline human demonstrations with a set of predefined proxy measures; second, the domain experts iteratively rank the quality of learned models based on the naturalness of the generated behaviors. Through this learning-correction loop, the generated behavior for NPC is guaranteed to be natural. (2) developing generative models with heterogenous inputs, i.e., the generated distribution should capture the underlying distributions and the joint distributions of data from different modalities [2]. We propose a generative AI that accepts both text and visual demonstrations for NPC to produce the target text outputs and the desired behavior. We will develop a unified generative model for multi-modal, multi-task and multi-embodiment inputs.
[1] Imitating Human Behaviour with Diffusion Models (ICLR 2023)
[2] Uni[MASK]: Unified Inference in Sequential Decision Problems (NeurIPS 2022)
Person specification
For information
- Candidates must hold a minimum of an upper Second Class UK Honours degree or international equivalent in a relevant science or engineering discipline.
- Candidates must meet the School's minimum English Language requirement.
- Candidates will be expected to comply with the University's policies and practices of equality, diversity and inclusion.
Essential
Applicants will be required to evidence the following skills and qualifications.
- This project requires mathematical engagement and ability substantially greater than for a typical Computer Science PhD. Give evidence for appropriate competence, as relevant to the project description.
- You must be capable of performing at a very high level.
- You must have a self-driven interest in uncovering and solving unknown problems and be able to work hard and creatively without constant supervision.
Desirable
Applicants will be required to evidence the following skills and qualifications.
- You will have good time management.
- You will possess determination (which is often more important than qualifications) although you'll need a good amount of both.
General
Applicants will be required to address the following.
- Comment on your transcript/predicted degree marks, outlining both strong and weak points.
- Discuss your final year Undergraduate project work - and if appropriate your MSc project work.
- How well does your previous study prepare you for undertaking Postgraduate Research?
- Why do you believe you are suitable for doing Postgraduate Research?