Sorry, the code for the "Library" window size was changed to work like dialogs in the Settings/Preferences page, which have a minimum width and height, as part of a fix for window content not resizing properly. I suspect it was not tested by enough people to know that some people find the window to be too large now. There is work under way to change this, but I don't know how soon Firefox will get updated (maybe in December? maybe in January??).
Until then, there is a method to work around it if it's enough of an irritant to you. Please see this thread for more information: