Self-optimizing RAG system with SLM for domain specific learning

Abstract

This research presents a Self-Optimizing Retrieval-Augmented Generation (RAG) system integrated with Small Language Models (SLMs) to support domain-specific learning. The system is designed to transform structured and unstructured content, such as domain-related documents, lecture recordings, and scanned notes, into a searchable, intelligent knowledge base. By leveraging lightweight and efficient SLMs, the solution offers cost-effective and scalable AI assistance tailored to specific subject areas. The system enhances traditional RAG architectures by incorporating reinforcement learning from human feedback (RLHF) to enable self-optimization. User feedback dynamically improves both the retrieval quality from vector databases and the response generation from the language model. In addition to interactive question-answering, the system provides personalized learning paths, curated reference materials, and progress insights.