This is the repository for Knowledge Editing for Multi-Hop Question Answering Using Semantic Analysis, which is published at the 2025 International Joint Conference on Artificial Intelligence (IJCAI 2025).
The MQuAKE datasets are provided in the /datasets/ folder. All CHECK prompts are provided in the /prompts/ folder. Embedding functions used by CHECK are in contriever_function.py. The relationship typing used by CHECK for the MQuAKE dataset is in mquake_relationships.txt. Other typing files are need to be created for other datasets. The CHECK Knowledge Editing framework is provided in CHECK.py. A working example of CHECK is given in CHECK_example.py.
The following command will run the example script:
python3 CHECK_example.py --model_name gpt-j --ds_name mquake-cf-3k --ds_start 0 --ds_end 10 --num_new_tokens 50 --similarity cos --sim_thresh 0.8 --use_gpu --gpu_num 0 --verbose
Explanations of all command line arguments are provided in example script. The values for num_new_tokens, similarity, and sim_thresh are the recommended parameters for experimental evaluation.
If you come across any bugs or issues with the code, or would just like to ask questions about CHECK or generally discuss Knowledge Editing, please feel free to email Dominic Simon at dominic.simon@ufl.edu or open an issue on this repository.
If you use our code or find the work helpful, please consider citing our work:
@inproceedings{simon2025check,
title = {Knowledge Editing for Multi-Hop Question Answering Using Semantic Analysis},
author = {Simon, Dominic and Ewetz, Rickard},
booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, {IJCAI-25}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
editor = {James Kwok},
pages = {8241--8249},
year = {2025},
month = {8},
note = {Main Track},
doi = {10.24963/ijcai.2025/916},
url = {https://doi.org/10.24963/ijcai.2025/916},
}