no-4

Harness engineering for AI agents: a conceptual framework and empirical evaluation in university information technology infrastructure

Authors:
Le Loc Xuan
Pages:
0
View:
12
Position:
2/2
Download:
6
This study proposes a conceptual framework for harness engineering for artificial intelligence (AI) agents based on large language models (LLMs), with the aim of clarifying the role of the infrastructure layer between model reasoning capabilities and the operational requirements of real-world information technology (IT) environments. The study employs an exploratory sequential mixed-methods design comprising three stages: a systematic scoping review of 147 publications published between 2021 and 2026, benchmark experiments involving three harness configurations across five evaluation suites, and semi-structured interviews with 12 IT infrastructure experts from universities in Vietnam and several Southeast Asian countries. The synthesis identifies six functional pillars of harness engineering: instruction rules, automated sensors, agentic loops, external memory, access control, and system observability. Across all five benchmarks, the full harness configuration demonstrated higher...
This study proposes a conceptual framework for harness engineering for artificial intelligence (AI) agents based on large language models (LLMs), with the aim of clarifying the role of the infrastructure layer between model reasoning capabilities and the operational requirements of real-world information technology (IT) environments. The study employs an exploratory sequential mixed-methods design comprising three stages: a systematic scoping review of 147 publications published between 2021 and 2026, benchmark experiments involving three harness configurations across five evaluation suites, and semi-structured interviews with 12 IT infrastructure experts from universities in Vietnam and several Southeast Asian countries. The synthesis identifies six functional pillars of harness engineering: instruction rules, automated sensors, agentic loops, external memory, access control, and system observability. Across all five benchmarks, the full harness configuration demonstrated higher reported accuracy and better post-failure recovery than the default configuration in the descriptive comparisons. Interview analysis indicates that the most prominent barriers to implementation in universities are a lack of technical expertise (10/12, 83.3%), incompatibility with legacy infrastructure (9/12, 75.0%), and cybersecurity concerns (8/12, 66.7%). The findings suggest that harness engineering should be treated as an engineering layer distinct from model optimization and implemented through a phased roadmap supported by internal capacity building, reference architectures, and appropriate governance and monitoring mechanisms.
Relate

Vinh University journal of science

Tạp chí khoa học Trường Đại học Vinh

ISSN: 1859 - 2228

Governing body: Vinh University

  • Address: 182 Le Duan - Vinh City - Nghe An province
  • Phone: (+84) 238.3855.452 - Fax: (+84) 238.3855.269
  • Email: vinhuni@vinhuni.edu.vn
  • Website: https://vinhuni.edu.vn

 

License: 163/GP-BTTTT issued by the Minister of Information and Communications on May 10, 2023

Open Access License: Creative Commons CC BY NC 4.0

 

CONTACT

Editor-in-Chief: Assoc. Prof., Dr. Tran Ba Tien
Email: tientb@vinhuni.edu.vn

Deputy editor-in-chief: Assoc. Prof., Dr. Phan Van Tien
Email: vantienkxd@vinhuni.edu.vn

Sub-Editor: Dr. Do Mai Trang
Email: domaitrang@vinhuni.edu.vn

Editorial assistant: Msc. Le Tuan Dung, Msc. Phan The Hoa, Msc. Pham Thi Quynh Nga, Msc. Tran Thi Thai

  • Address: 4th Floor, Executive Building, No. 182, Le Duan street, Vinh city, Nghe An province.
  • Phone: (+84) 238-385-6700 | Hotline: (+84) 97-385-6700
  • Email: editors@vujs.vn
  • Website: https://vujs.vn

img