Imagine a manufacturing floor where technicians can get instant, accurate guidance on complex tasks, just by asking questions. That’s the vision behind new research exploring how Large Language Models (LLMs) can revolutionize manufacturing processes. The challenge? Current LLMs often struggle with the nuances of real-world tasks, particularly in specialized fields like manufacturing. Researchers have developed a novel approach called QA-TOOLBOX, a conversational question-answering system designed to provide process task guidance. This system uses a combination of specification documents, real-world task executions, and technician narrations to create a comprehensive knowledge base. One of the key innovations is data augmentation using LLMs. Since gathering real-world manufacturing data can be difficult and sensitive due to intellectual property concerns, the researchers cleverly used LLMs to generate synthetic data that mimics real interactions. This augmented dataset, along with real technician questions, helps train the QA system to understand the specific language and needs of manufacturing tasks. Another hurdle is evaluation. How do you measure the effectiveness of an AI assistant when there’s no single right answer? The researchers employed an “LLM-as-a-judge” approach, using a separate LLM to evaluate the responses generated by the QA system. Interestingly, expert validation showed that the LLM judge provided more reliable ratings than traditional crowd-sourced evaluations. While the research demonstrates promising results, challenges remain. Current LLMs struggle with long-term visual context, crucial for understanding complex processes. Future research will focus on incorporating multimodal information, such as video demonstrations, to provide richer guidance and improve the overall efficiency of AI assistants in manufacturing. The development of QA-TOOLBOX opens exciting possibilities for the future of manufacturing, promising to empower technicians, streamline processes, and ultimately improve productivity.