Large language models (LLMs) are impressive, but they sometimes make things up – especially when tackling complex reasoning. This "hallucination" problem limits their use in areas demanding high reliability. New research introduces FiDeLiS, a clever method to make LLMs more truthful by connecting their reasoning to verifiable facts within knowledge graphs (KGs). Think of KGs as vast libraries of structured information. FiDeLiS uses a keyword-enhanced retrieval system to pull relevant entities and relations from these KGs, creating a sort of roadmap for the LLM to follow. Instead of wandering aimlessly and potentially hallucinating, the LLM uses this roadmap to construct and refine reasoning paths, ensuring every step is backed by solid facts. What sets FiDeLiS apart is its use of "deductive verification." At each step, the LLM checks if its reasoning holds water logically. This careful, step-by-step verification prevents the LLM from going off-track and making things up. Experiments show FiDeLiS outperforms existing methods, even those requiring extensive training. It's faster, more reliable, and doesn't need retraining for new tasks. This research is a big step toward making LLMs more trustworthy and opens exciting possibilities for using them in fields like healthcare and scientific research, where accuracy is paramount.