Ontology Informed Machine Learning for Computer Vision

Project description

This Ph.D. project will explore how we can make machine learning models aware of background knowledge in computer vision tasks, in the form of ontologies, with the aim of improving their performance, reducing the amount of training data required, increasing their robustness, and increasing their transparency.

Take to understand visual scenes as an example. A first step in scene understanding is the detection and recognition of objects from the input images, such as "person" or "bicycle", which is well-understood. A second step is the detection of visual relations between objects such as "person riding a bicycle" in the images, which is currently far more challenging. Machine learning models, such as neural networks, can in general be used to construct a mapping from an input image to a set of semantic relationships (or a scene graph) between the detected objects. However, success of state-of-the-art scene relation/graph extraction systems relies on the use of a very large amount of labeled image examples, called "ground truth". We could use an ontology to inform the vision model that a bicycle is a device with 2 wheels, and that a person can ride or push a bicycle. This project will build on existing work in the area (in particular a prior PhD project supervised by the same team, partially described in [1]) and investigate how the rich knowledge in an ontology can be injected into machine learning modelling.

This project combines the fields of machine learning, computer vision, and logic-based knowledge representation (ontologies and description logic), and aims to bring symbolic and neural approaches to AI together.

We will consider applicants who have:
* Some existing knowledge in some of the above fields, e.g., deep learning, computer vision, or ontologies and description logic.
* Enthusiasm for research across the above research fields.
* An excellent undergraduate degree in Computer Science or Mathematics (or related discipline), and preferably, a relevant M.Sc. degree.
* Very good experience with computer programming of mathematical models and algorithms (in Python, Matlab or other platforms).
* A willingness to investigate complex research problems carefully and to communicate the results of these investigations clearly both in spoken and written form.
* Enthusiasm for being part of the CS research school and engaging with its student and staff members.
*Some research experience, e.g., evidenced by well-written publications or project report, would be a plus.

Please, note that applicants must additionally satisfy the standard requirements for postgraduate studies at the University of Manchester, such as a first-class or high upper-second class (or an equivalent international qualification) and English language qualifications, as stated in the PGR guidelines.

Qualified applicants are strongly encouraged to informally contact the supervising academics Dr. Tingting Mu (tingting.mu@manchester.ac.uk) or Prof. Uli Sattler (Uli.Sattler@manchester.ac.uk) to discuss the application and possible research directions prior to applying.

[1] Mirantha Jayathilaka, Tingting Mu, Uli Sattler, Towards Knowledge-aware Few-shot Learning with Ontology-based n-ball Concept Embeddings. ICMLA, 292-297, 2021.