I have several logins that somehow have been duplicated in 1Password 8. This may have happened in different times when I've attempted to migrate my 1Password 7 data to my 1Password.com account. In any case, I have a load of duplicates in 1Password 8 and it would be very helpful if there were an efficient tool or feature to help me merge/dedupe these entries (and ideally test them to make sure that I'm preserving the correct credentials). Any thoughts/ideas for how to handle this situation? 1Password Version: 1Password for Mac 8.5.0 80500017, on BETA channel Extension Version: 2.1.4 OS Version: macOS 12.0.1 (21A559)

Hello everyone! 👋 With the release of the latest beta for Mac, Windows, and Linux you can now see a list of your duplicate items in the app. Choose “Items with duplicates” in your Watchtower dashboard to see or delete all of your duplicates. Make sure that you're running the latest beta to see the feature. As always, please let us know if you find the tool useful and if you have any suggestions for the team. 🙂 -Dave

My Duplicates occurred when I upgraded from V7 standalone on my least used machine to V8. I was already using V8 on two other Macs an iPhone and two iPads. That upgrade process was terrible. It wanted me to create a new 1Password membership (I already have a membership which there was no obvious way to connect to.). I ended up downloading V8 and installing it. It said I had to convert my V7 data to V8. It was not at all clear that I could have just skipped that step. Now I've got about 800 duplicates. The upgrade process needs to be more clear so duplicates are not created in the first place. We also need a duplicate finder/remover tool.

In an ideal world, nobody will duplicate data at import time, but actually it will happen. I will not discuss how it can happen or can be avoided, only that it will happen - proof is this thread and others. Duplicate items are not always user error. Switching to a decent password manager often means merging from multiple sources, then consolidating: import from previous password manager plus Chrome's plus Firefox's plus Safari's internal password managers. One does import once directly after account creation, then never again. You at Agilebits do it never again, so you don't see really any need for postprocessing aid. However, for a new user it's crucial, it's a barrier. With duplicate items I mean login items that contain the same userid+password combination. What I propose you can do about this: - give an overview how many items match above description, in a similar way Watchtower does this. Also create filters, so the user can see a list of only matching entries. - also give an overview how many items match with url+userid, but different password. These might represent current and obsolete entries of the same login. The category could be something like "unable to merge", "merge conflict" or "password mismatch". Also make a filter for these. - explain if duplicates are distributed over multiple vaults. If 2 vaults contain the same, because the user imported the same 2 times, the user should see this and the easy fix for the user is to just delete one of these vaults - make it clear over how many different vaults how many duplicated entries are distributed - add the opportunity to "merge" duplicate entries in one vault, in spirit similar to functions to merge different address entries in address books in other apps. Merge only items within the same vault. - this merging would be available from the context menu you get if you select multiple entries. Merging candidates are only taken from the entries the user selected. - This merging functionality could find login entries with same userid+password combination, then choose the oldest entry as "merged" entry. Then merge all URLs from the other entries to this entry, the same with any custom fields and sections with different values (don't add custom fields with same title+value multiple times). And merge the password history of course. Also skip merging for history entries with identical password+timestamp combinations. After merging, add the tag "merged " to the merged entry (so the user can find all processed entries and can check if all is ok). Then delete the items their content was merged into the merged item. It's an investment in getting new users. I bet the turnover rate from ex-Lastpass users would be higher if you had such a functionality. When I was new, I merged entries from Password Safe and Chrome. I almost gave up, because it was so tedious - after all, Chrome autofill wasn't that bad, and Password Safe for just a local vault also wasn't that bad for all non-browser login items. A month after the trial expired I gave it a second shot and decided to do the tedious work.

Tool for detecting and removing duplicates? [Available in 8.10.18]

andrew_l_1P
1Password Team
3 years ago
Hi @mikentosh,

The hope is that we can lessen the instances where duplicates are created in the first place, but I can still understanding wanting the tool in the latest version of 1Password, I'll add your feedback to the tracker mentioned earlier in this thread.

If you haven't already, I recommend contacting us via email at support@1Password.com so we can investigate what caused your duplicate items. You can include a link to this thread and your forum username so we can connect the conversations. Any additional information like a recent migration from an earlier version or importing data will also help! The tool in 1Password 7 is very conservative so depending on the source of the duplicate items, it may not be a worthwhile course of action.

Let me know if you have any questions. 🙂

ref: IDEA-I-615
Former Member
3 years ago
Thanks, Andrew.

I have contacted via the support email address and have received a couple of responses and guidance on how to delete items. I completely understood how to do that prior to reaching out. Even deleting multiple items at once. I suppose that will have to suffice for now.

I have migrated through multiple versions of 1Password over the years. I’ve been a fan since day one. I don’t recall having this much of an issue until Version 8. I think I might go back and check my V7 database and compare. I can still install that, correct?
andrew_l_1P
1Password Team
3 years ago
Hi @mikentosh,

Glad to hear you have a path forward, sorry it wasn't as seamless as we would've hoped though.

That's correct, 1Password 7 remains available as a legacy download on our website: 1Password Downloads

Some of the duplicate issues we saw were a result of lingering local vaults from previous versions that weren't deleted, so the same thing won't happen with future versions now that folks are using 1Password.com accounts. I do recommend deleting old local vaults if you still have them though, as to avoid any already-migrated data from being sent over to 1Password 8 again!
Former Member
3 years ago
+1 please. I too have a boatload of duplicates since migrating from a local legacy 1P vault file to 1P online.
Former Member
3 years ago
OK, so 4 months later. Where's the de-duplication feature? Just moving from LastPass because they are clowns, and now find that my import generated a bunch of duplicates (over 800 pairs). How do I clean this up?
DeppHeard
New Contributor
3 years ago

OK, so 4 months later. Where's the de-duplication feature? Just moving from LastPass because they are clowns, and now find that my import generated a bunch of duplicates (over 800 pairs). How do I clean this up?

Not 4 months, more like 13 months at least https://1password.community/discussion/125331/tool-for-detecting-and-removing-duplicates/p1
andrew_l_1P
1Password Team
3 years ago
Hi @et01267,

Thanks for your feedback and welcome to 1Password!

I don't have any updates regarding a duplicate tool at the moment, but if you're seeing duplicate items from a repeat import on an otherwise empty account and you still have the source file, I recommend creating a new vault and importing your data again.

If your duplicates are in a user-made vault you can replace that vault with the new one. If your duplicates are in your Private vault you can delete its contents and then move the entirety of the new vault into your Private vault.

I recognize this is only viable for new accounts where re-importing is an option though so I'll add your feedback and use case to our internal discussion.

Let me know if you have any questions. 🙂

ref: IDEA-I-615
Former Member
3 years ago
In my case, I have noticed that the duplication is actually in the exported LastPass passwords file. I went through that file line by line to remove the duplications before uploading to 1Password.
ctphoto
New Contributor
3 years ago
Perhaps for the new year 1Password will actually fix this. We can dream......
Former Member
3 years ago
andrew_l_1P i have just opened a 1p account as i am considering moving from lastp... (unmentionable) i have a family account and i moved everything across and i have 3 duplicates for every entry , so over 4000 entires in my single vault , I have yet to load the data for the rest of the families accounts in light of this.
we are now in 2023 and i see the lack of a duplicate tool has been hanging around here for over 8 months now like a bad smell, is there any light at the end of these requests for a duplicate tool or should i just move on ?

Forum Discussion

Tool for detecting and removing duplicates? [Available in 8.10.18]

Recent Discussions

'Read-only user' mode: Is there one?

Safari extension setup screens keep appearing

Usability Digest May 2024: Autosave, Sharing, and Sign-in Experience

Cryptomator integration

How to export a single vault