Despite Gandalf visiting Dol Guldur and encountering The Necromancer in The Hobbit: An Unexpected Journey, there were still a few reasons why the White Council didn’t listen to him about Sauron’s return, even if it made them appear naive (or even silly) to fans. At the time, Saruman was the only other wizard/Istari in Lord of the Rings on the White Council with Gandalf, and led other powerful members like Elrond and Galadriel and convened to discuss important matters in Middle-earth. Each of them had lived through Sauron’s depravity and had been affected in some way, making it particularly strange why they didn’t take Gandalf’s warnings about a shapeshifter in the ruins more seriously.

https://screenrant.com/lord-of-the-rings-white-council-not-attack-sauron-why/