Project description

Large Language Models (LLMs), such as the GPT series and the Llama series, have achieved great success in NLP, AI and many domain applications. However, they still suffer from several issues such as hallucination, lacking domain or updated knowledge, and the shortage of explanation and complex reasoning capabilities. Although state-of-the-art reasoning LLMs like DeepSeek-R1 have made progress for reasoning, they are still not sufficient for constructing explainable and trustful intelligent systems, especially for many domains requiring high safety. Meanwhile, as the fast development of LLM techniques, their efficient, comprehensive and automatic evaluation, especially on addressing these critical issues, becomes urgent.

Traditional knowledge base as well as symbolic reasoning techniques, provide a good opportunity to address the above mentioned issues of LLMs, due to their high transparence, decisiveness and rich resources built in the past decades. In this project, we will utilise Knowledge Graphs, which includes multi-relational graphs composed of factual data like Wikidata, those ontologies that represent conceptual knowledge and logical relationships such as SNOMED CT and GO, as well as their combinations, for the following research topics: (1) evaluating the domain knowledge, reasoning capability and explainability of existing and future LLMs (see [1] and [2] for two previous example studies); and (2) enabling the explainability of LLMs in applications like question answering and fact verification, with external knowledge and symbolic reasoning capabilities of Knowledge Graph.

The Information Management Group at the University of Manchester invites applications for PhD candidates in the area of Neural-symbolic AI with Knowledge Graph. PhD projects in this area will explore how contemporary techniques in Machine Learning, NLP and Symbolic AI (such as Knowledge Representation and Reasoning) can be used as a foundation to address the challenges of the above two research topics on LLMs.

Examples of research challenges include: 1) how to automatically construct benchmarks from logic-equipped and/or large-scale knowledge graphs for complex evaluation tasks; (2) how to design quantitative experiments for accurately assessing the specific capabilities of LLMs; (3) how to retrieve relevant knowledge, infer justification and proofs for LLM explainability; (4) how to align LLMs with specific logical rules and enable LLMs for logical reasoning.

Applicants are expected to have:

1. An excellent undergraduate degree in Computer Science or Mathematics (or related discipline), and preferably, a relevant M.Sc. degree.
2. Confidence and independence in programming complex systems in Java or Python.
3. Previous academic or industry experience in at least one of the relevant topics such as Machine Learning, NLP, Knowledge Representation and Reasoning, Semantic Technology.
4. Excellent report writing and presentation skills.

Please note that applicants must additionally satisfy the standard requirements for postgraduate studies at the University of Manchester, such as a first-class or high upper-second class (or an equivalent international qualification) and English language qualifications, as stated in the PGR guidelines.

Qualified applicants are strongly encouraged to informally contact Jiaoyan Chen (jiaoyan.chen@manchester.ac.uk) or Uli Sattler (uli.sattler@manchester.ac.uk) to discuss the application prior to applying.

[1] He, Y., et al. "Language Model Analysis for Ontology Subsumption Inference." Findings of the ACL 2023.
[2] Hu, Nan, et al. "Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphs." ACL 2025.