All banshee libraries can be found in the directory 'libraries'. Some of them are just a collection of functions, others are PHP classes. The tool 'show_library_api' can be used to show the API of a library.
Core libraries should not be removed from the installation or be changed, unless you know what you are doing. Required libraries are needed by one or more modules or administration pages. Optional libraries are... wel... optional.