Large language models (LLMs) are transforming how we search for information. But how do these powerful AI models actually decide what's relevant? New research probes the inner workings of ranking LLMs, revealing surprising insights into the features they prioritize—and the ones they ignore. The study focused on RankLlama, a model fine-tuned for ranking passages, and used a technique called probing to analyze its neural activations layer by layer. The researchers discovered that RankLlama prioritizes several human-engineered features, like the number and ratio of covered query terms, confirming that some traditional ranking wisdom still holds true in the LLM era. More interestingly, they found strong evidence that the model also considers combinations of features, suggesting LLMs learn complex relevance patterns that go beyond simple keyword matching. However, the study also highlighted some blind spots. RankLlama seemed to overlook certain features, such as sum and max of tf*idf scores, which have been important in previous ranking models. This discrepancy raises questions about how LLMs prioritize information differently than traditional approaches and hints at potential areas for improvement. Moreover, when tested with out-of-distribution data—queries and documents unlike those it was trained on—RankLlama 13b showed signs of overfitting, prioritizing features like stream length that don't generalize well to unseen data. This highlights a key challenge in LLM development: ensuring they can generalize their learned knowledge to diverse information landscapes. The research underscores that while LLMs represent a paradigm shift in search, understanding their decision-making processes is crucial for refining their performance and ensuring they deliver truly relevant results. This work opens exciting avenues for future research, including the development of new features tailored for LLMs and the creation of more robust and interpretable ranking models.