Sorry, there's no way currently to remove that button. It will be automatically hidden once you visit a page, and will only show up when you are searching.
It is designed to give you easy access to your search engines, so you can switch between them easily when you want to search.
