Avoid duplicate bookmarks(saved urls)

[Anonymous]

feature name: Avoid duplicate bookmarks(saved urls)

feature description: 

1. Whenever the user saves a url the Mozilla Firefox browser checks whether the saved url is a duplicate or not.
2. An alert appears if the url is duplicated. If not, the alert does not appear.

Example

Expected benefit

1. Avoid repeated bookmarks
2. Data consistency

Reference

1. https://sweetalert.js.org/docs/

[eduardojuan]

A tool to search for and remove duplicate bookmarks

I don't know where duplicate bookmarks come from but they are a pain. Removing them is torture.

Why don't you add a tool to search for and remove duplicate bookmarks and folders? Windows utilities use this approach to remove duplicate files in the computer.

Please do it, or tell me of an expeditious alternative to remove mine. I have hundreds of unwanted duplicate bookmarks! Example is attached.

Also, can you explain where they came from?

PS/ I AM NOT A NEW MEMBER. I HAVE BEEN USING FIREFOX FOR 20 YEARS!

[Anonymous]

@eduardojuanHi! How are you? I had this same idea here, I hope to help: https://connect.mozilla.org/t5/ideas/avoid-duplicate-bookmarks-saved-urls/idi-p/4625

Source-code

```

CREATE TABLE IF NOT EXISTS moz.bookmarks(

moz_places.id INTEGER PRIMARY KEY,
moz_bookmarks.id INTEGER NOT NULL,
moz_bookmarks.fk FOREIGN KEY (moz_places.id) REFERENCES moz.bookmarks(moz_places.id),
moz_places.guid TEXT NOT NULL,
moz_bookmarks.guid TEXT NOT NULL,
moz_bookmarks.title TEXT NOT NULL,
moz_places.url TEXT NOT NULL UNIQUE,
moz_places.title  TEXT NOT NULL,
moz_places.description  TEXT NOT NULL,
moz_places.preview_image_url TEXT NOT NULL,
moz_places.visit_count INTEGER NOT NULL,
moz_annos.content TEXT NOT NULL,
datetime(moz_places.last_visit_date/1000000,'unixepoch','localtime') as 'Last',
datetime(moz_historyvisits.visit_date/1000000,'unixepoch','localtime' ) as ' LastVisit',
datetime(moz_annos.dateAdded/1000000,'unixepoch','localtime') as 'BookMarkAdded', 
datetime(moz_annos.lastModified/1000000,'unixepoch','localtime') as 'BookMarkModified',
moz_places.typed as 'P_Typed',
moz_hosts.typed as 'H_Typed',
moz_inputhistory.input,
moz_inputhistory.use_count,
moz_anno_attributes.name as 'type',
hex(moz_places.url_hash) as 'hash'
);

```

Notes

1. I wanted Mozilla Firefox to have this feature to identify duplicate links, I made a sqlite that demonstrates this, where duplicate registry of URLs are not allowed.
2. I do not know if Firefox still uses the 'places.sqlite' file, old versions of Mozilla Firefox used the files 'places.sqlite' to insert and update bookmarks.
3. If you want to read everything I researched, the links were below

References

1. https://github.com/Sean-Der/WebSQL-For-FireFox
2. https://stackoverflow.com/questions/8129938/websql-in-firefox
3. https://www.acquireforensics.com/services/tech/mozilla-firefox.html
4. https://wiki.mozilla.org/Places/Places_SQL_queries_best_practices
5. https://stackoverflow.com/questions/52343272/sql-query-on-tags-in-places-sqlite-of-firefox
6. https://wiki.mozilla.org/Places/AsyncAPIsForSync#Bookmarks
7. https://github.com/phhu/browserSearchShortcuts
8. https://github.com/Dissimilis/BookmarksManager
9. https://github.com/phhu/browserSearchShortcuts
10. https://gist.github.com/hyperreal64/0cfc47dde15e4ff9ec95fee3a90bf619
11. https://gist.github.com/Alligator-1/3c49a048eb77f6b8ad4baaeef8bccccc
12. https://connect.mozilla.org/t5/ideas/show-the-keyword-field-when-creating-bookmark/idi-p/3441
13. https://connect.mozilla.org/t5/ideas/duplicate-bookmarks/idi-p/4365
14. https://stackoverflow.com/questions/464516/firefox-bookmarks-sqlite-structure
15. https://wiki.mozilla.org/images/d/d5/Places.sqlite.schema3.pdf
16. https://support.mozilla.org/en-US/kb/bookmarks-firefox
17. https://sleeplessbeastie.eu/2018/05/07/how-to-import-firefox-bookmarks-to-nextcloud-application/
18. https://www.aplawrence.com/Web/firefox-sqlite.html
19. https://forums.mozillazine.org/viewtopic.php?f=38&t=2133167
20. https://github.com/stark4n6/Forensic-SQL-Queries
21. https://github.com/kacos2000/Queries
22. https://gist.github.com/iafisher/d624c04940fa46c6d9afb26cb1bf222a
23. https://stackoverflow.com/questions/6175924/simple-bookmark-manager-in-sqlite
24. https://kb.mozillazine.org/Places.sqlite
25. https://www.sqlitetutorial.net/sqlite-create-table/

[Sbernecchia]

I sometime create duplicate bookmarks on purpose (in different folders).

So adding duplicates bookmarks should not be an error, but only an (optional) warning.

[Anonymous]

@SbernecchiaHi! How are you? Thank you for feedback!

[Frijheid]

Strongly agree

I think the warning when placing duplicate bookmarks will help the user in adding bookmarks, the main goal is not to have duplicate bookmarks

[Frijheid]

Strongly agree.

I also think the warning when placing duplicate bookmarks will help the user in adding bookmarks, the main goal is not to have duplicate bookmarks.

[metropical]

so that one can eliminate dupes and/or merge

[Jon]

Thanks for submitting an idea to the Mozilla Connect community! Your idea is now open to votes (aka kudos) and comments.

[Jon]

Similar idea here: Avoid duplicate bookmarks(saved urls)

[c0d3h4x0r]

Absolutely desperately needed and long overdue!  Firefox stupidly and obnoxiously re-imports the same bookmarks from other browsers over and over again, creating countless duplicates which are unbelievably painful to clean up.  Ideally, Firefox would stop re-importing the same bookmarks and creating duplicates in the first place!

Here's how it happens:

1. Context: Existing Chrome or Edge user with tons of established bookmarks, using that browser across multiple PCs, with them all configured to sync bookmarks using a Google or Microsoft account.
2. User installs Firefox on first PC.  Setup/first-run imports bookmarks from Chrome or Edge.  Now all the bookmarks are in Firefox, as expected.  User sets up Firefox account so their bookmarks will sync to Firefox on other PCs/devices.
3. User installs Firefox on second PC.  Setup/first-run imports the same (sync'd) bookmarks from Chrome or Edge.  Now all the bookmarks are in Firefox, as expected.  User sets up Firefox account so their bookmarks will sync to Firefox on other PCs/devices.
4. Ooops!  Now the user's Firefox account contains two identical sets of imported bookmarks -- so they get merged, and now the user has duplicated bookmarks in Firefox, which of course sync across both PCs running Firefox.  BAD!
5. User installs Firefox on third PC.  Setup/first-run imports the same (sync'd) bookmarks from Chrome or Edge.  Now all the bookmarks are in Firefox, as expected.  User sets up Firefox account so their bookmarks will sync to Firefox on other PCs/devices.
6. Ooops!  Now the user's Firefox account contains three identical sets of imported bookmarks -- so they get merged, and now the user has triplicate bookmarks in Firefox, which of course sync across all three PCs running Firefox.  VERY BAD!

This kind of nonsense is unbelievably absurdly awful!  Please add a feature to help users clean up this disaster, and simultaneously, please fix your "import from other browsers" feature to stop reimporting the same bookmarks that have already been imported on another PC already!

This really shouldn't be difficult to get right.  Just add some additional attributes to your bookmarks.html format to keep track of the fact that a bookmark was originally imported from another browser and should thus never be re-imported again.  Or at the very least, when importing bookmarks, always import them into a uniquely-named folder structure ("Imported Bookmarks/Edge/2021-04-05-221736", "Imported Bookmarks/Chrome/2022-12-20-061827", etc) to keep duplicates separate for easy manual deletion later.

[Lebel]

Compare folders (and sub folders) content to remove duplicates

Hi,

There's some add-on to remove duplicates but it's "All or nothing" sometime (everytime in my case) you just want to remove duplicates in specifics folders and not in all your bookmarks. Moreover it's difficult to chose the folder where you want keep the bookmark and the others you don't and sometime you want to keep several time the same bookmark.

It could be very useful to compare folders (and sub folders) to removes duplicates. With the possibility to indicate the one where you want to keep the duplicate and the one you don't.

[Zarragossa]

1. Make sure that sync is working better then it does so far. I fixed all the bookmarks on one of my systems and one day later again the whole bookmark database is messed up badly with 100-1000=ands of duplicates. I have spent endless hours in fixing my bookmark which have become useless indeed.

2. Have a duplicate remover tool that removes all bookmark in all folder that are duplicates or at least remove all duplicates in each folder. Maybe some person wants two of the same bookmarks in two different folders. Right now it is as bad as it gets, especially because sync works horribly. It has to work multi directional especially if you offer in iCloud of Apple to sync with Safari as well. You have to work on that with Apple as well. At the moment I give Firefox a 2 out of 100 and Safari at best a 1 out of 100. I asked for this for 10 years plus and nothing ever gets better.

[Jon]

(Note: similar ideas have been merged into this thread)

[Zarragossa]

I have no idea what you are saying if anything I see nothing at all. I resent the fact that I have organized my bookmarks in Firefox and Safari 100 times already and a day later they are messed up again and I have up to 100 duplicates per website.

I tried to move a few bookmarks into another folder for instance and the system used 123 GB of memory and I had to force quit the app. Ridiculous software indeed and I bought over 700 software packages in my time. 

The way Safari and Firefox deal with bookmarks is under the lowest level of programming I know. Both are crappy browsers if they can't even handle the bookmarks. What purpose do bookmarks have if any? I need to go back to a previous website I liked hence I have made a bookmark but if I have over 10000 duplicates how should I find my bookmark, it defeats its purpose.

[Zarragossa]

What good is a kudo if I don't get any answer to the problem. Waste of my time