Call for Evaluation Challenges

IJCKG 2025 will organize evaluation challenges related to Knowledge Graph, aiming to provide researchers with a platform to test technologies, algorithms and systems. Evaluations challenge organizers of IJCKG 2025 can select the platform and evaluation plan by themselves, and we sincerely solicit evaluation challenges from researchers, research institutions and enterprises in related fields.

At IJCKG 2025, each evaluation challenge will receive one slot during the main conference, where organizers present the challenge, and participants present the solutions. At least one organizer must register and be present at the conference. Winners will receive a certificate and be invited to present their systems during the poster and demo session.

Topics
For IJCKG 2025, evaluation challenge proposals are invited for all tasks on Knowledge Graphs, including but not limited to:

  • Knowledge Representation and Reasoning
  • Knowledge Acquisition and Knowledge Graph Construction
  • Knowledge Graph Querying and Management
  • Semantic Web and Data Mining
  • Natural Language Understanding and Semantic Computing
  • Question Answering and Semantic Search
  • Neuro-Symbolic AI
  • Large Language Model and Knowledge Graph
  • Machine Learning and Knowledge Graph

Proposal Submission Guidelines
Proposals for evaluation challenges should be concise and include:

  • Names and affiliations of the organizers
  • Description of the evaluation challenge, including the specific task to be addressed, the details of evaluation dataset, and its relevance to Knowledge Graph
  • Procedure for evaluating the performance of systems, including metrics and availability of evaluation software
  • Expected number of participants with supporting evidence
  • Please submit the proposal to the email: tianxingwu 'AT' seu.edu.cn

Timeline for Evaluation Challenge Organizers:

  • Submission of evaluation challenges: June 1, 2025
  • Website & first call for participation: June 7, 2025
  • Release datasets (including the test set): June 7, 2025
  • Submission of systems & system description papers: August 1, 2025
  • Notification of acceptance & winner announcement: August 10, 2025
The winners will have the opportunity to publish their results as a short paper
All deadlines are 23:59 AoE (Anywhere on Earth).

Evaluation Challenge Chair

  • Tianxing Wu (Southeast University, China)