-Title: Chain of Thought Based Reasoning Segmentation Using Text Image Multimodal LLM for Smart Logistics
-Journal/Conference: Scientific Reports
-Authors: Taewook Wi,, Seokhyun Gong, Suyeon Park , Minjun Jeong , Seunghwan Lee , and Jongpil Jeong
-DOI: 
-Journal/Conference Link: 

Abstract: This study proposes the MLLM Logi-flow framework, that integrates a Multimodal Large Language Model (MLLM) based reasoning segmentation approach to advance the automation of product classification and storage processes in warehouse management systems. Conventional strategies in logistics environments have primarily relied on physical characteristics such as box size and shape, which limit the system’s ability to perform fine-grained operations based on product-specific attributes. To overcome these limitations, this study designs a framework that enables the model to infer and segment complex product attributes, such as “fragile” and “perishable” for each individual item. To achieve this, a 3-step pipeline was developed using the MVTec D2S dataset, which reflects complex logistics environments, resulting in the construction of a reasoning segmentation training dataset comprising 503 samples. With only 243 training samples, we fine-tuned the existing LISA-7B and LISA-13B models to significantly improve segmentation performance based on product attributes. Furthermore, we introduce Logistics-CoT, a prompt generation method that incorporates the Chain-of-Thought (CoT) mechanism. Experimental results show that both LISA-7B and LISA-13B models achieved consistently better performance in terms of Complete Intersection over Union (cIoU) and Generalized Intersection over Union (gIoU) metrics when instruction tuning with Logi-CoT was applied, compared to simple fine-tuning alone. In particular, the attribute “perishable", which demands higher-level reasoning, exhibited clear improvements. These findings demonstrate that CoT-based reasoning prompt design effectively enhances segmentation performance for complex attribute classification in logistics environments.

-Status: Submitted (2026/07/03)