I agree with you. But for now, this is what you can do:Start > All > Scroll to 'Firefox web apps' folder > Right-click a web app > More > Open file location.You'll be able to rename or delete the shortcuts.
I've heard that the feature will be available in stable starting with version 142, but disabled. Do you have any comments on this? I've noticed that it's already available on the stable build, but not in a working state...