A powerful defender.
A library with the attributes you describe would be a natural stronghold for a powerful mage or cleric. Access to all those spells would eliminate the need to travel to find new spells and items. Occasional visitors who conveniently lose all memory of the location once they leave would be a good source of whatever the defender needs. Additionally, an ally like this who has a scrying stone or other means to communicate with the outside would be an invaluable resource to help the library defend itself.
If a symbiotic relationship can be established with a friendly ally, that would be great for the library. On the other hand, if this defender is evil and devotes him/herself to attacking and looting all visitors to the library, that could provide even more plot fodder for the adventuring party seeking the library.
Along the same lines, if this raiding party does contain mages of a scholarly bent, one of those may see the advantage of making the library his home and decide to stay here, perhaps betraying his comrades in the process, to become that defender.
Depending on how deep you want to get into this idea, it's entirely possible the library could become home to a school of magic, as well. The synergies involved in this would be similar to the other options above.