석사과정 임재훈 학생(지도교수: 정종필)의 연구(mLLM-CRD: multi-modal LLM based context-aware rule detection using LoRA-integrated attention for worker safety monitoring)가 Journal of King Saud University - Computer and Information Sciences (Impact Factor: 6.1 (2024); 5-year Journal Impact Factor: 5.7 (2024))에 게재됐다.

https://link.springer.com/article/10.1007/s44443-026-00572-2 / https://doi.org/10.1007/s44443-026-00572-2

논문요약 - Large Vision-Language Models (VLMs), which jointly process visual and textual information, have demonstrated strong performance in general scene understanding; however, their application to safety-critical industrial environments remains limited by the need for precise semantic reasoning, interpretability, and computational efficiency. Conventional worker safety monitoring systems typically rely on rigid object detection pipelines, which are prone to error propagation and provide limited explanatory capability. This paper presents mLLM-CRD, a multimodal framework that formulates safety compliance assessment as an image-to-language reasoning task. Built upon Bootstrapping Language-Image Pre-training (BLIP-2), the proposed approach applies a joint parameter-efficient fine-tuning strategy using Low-Rank Adaptation (LoRA), which is integrated into both the Querying Transformer (Q-Former) and the encoder-decoder-based language model while keeping the vision encoder frozen, thereby enabling task adaptation with minimal additional parameters. To support rule-aware semantic reasoning, we construct a scenario-based question-answer (QA) dataset that encodes safety compliance conditions as structured language queries derived from industrial safety rules. Experimental results show that mLLM-CRD consistently outperforms zero-shot VLM baselines in both classification accuracy and natural language generation quality, particularly under class-imbalanced conditions. These findings suggest that unified multimodal reasoning can provide interpretable, language-based assessments of worker safety compliance, offering a practical alternative to conventional multi-stage detection pipelines in real-world industrial settings.