Tool for detecting and removing duplicates? [Available in 8.10.18]

I agree with Tertius, a Merge function would help a lot.

I'm one of those who have been asking for this to be fixed for NINE years. Thousands of duplicates introduced when migrating from standalone to Cloud, for 1Pass 6, before someone fixed the bug that prevented the problem. Early adopters were affected (and neglected) most.

The Team came close to fixing it with a tool in 1Pass 7. The tool was still too restrictive, but it got rid of some duplicates. It always felt as though any college programmer could fix the problem in an hour or two.
They removed the tool altogether when introducing 1Pass 8.

The Team has never been willing to share exactly why they won't allow us to modify the database entries in a list fashion, or in bulk.
I'm guessing the main reason is not because this is difficult to do, but rather they're afraid that divulging the format of the database, will give away some trade secret.

I am looking for a password manager app... but the lack of facility to remove duplicates is a non- starter... I'll keep looking.

+1

Thank you Tertius for the in-depth and considered post here. Your reasoning makes sense to me. I'll pass this on to our Product team immediately so that it can inform the process going forward, and of course will update here should we have more to share. Many thanks 🙏

Hi @peregrine, sorry that this happened in the switch. I have added a vote for the feature on your behalf. Thanks for letting us know why this matters for you.

In an ideal world, nobody will duplicate data at import time, but actually it will happen. I will not discuss how it can happen or can be avoided, only that it will happen - proof is this thread and others. Duplicate items are not always user error. Switching to a decent password manager often means merging from multiple sources, then consolidating: import from previous password manager plus Chrome's plus Firefox's plus Safari's internal password managers.

One does import once directly after account creation, then never again. You at Agilebits do it never again, so you don't see really any need for postprocessing aid. However, for a new user it's crucial, it's a barrier.

With duplicate items I mean login items that contain the same userid+password combination.

What I propose you can do about this:
- give an overview how many items match above description, in a similar way Watchtower does this. Also create filters, so the user can see a list of only matching entries.
- also give an overview how many items match with url+userid, but different password. These might represent current and obsolete entries of the same login. The category could be something like "unable to merge", "merge conflict" or "password mismatch". Also make a filter for these.
- explain if duplicates are distributed over multiple vaults. If 2 vaults contain the same, because the user imported the same 2 times, the user should see this and the easy fix for the user is to just delete one of these vaults
- make it clear over how many different vaults how many duplicated entries are distributed
- add the opportunity to "merge" duplicate entries in one vault, in spirit similar to functions to merge different address entries in address books in other apps. Merge only items within the same vault.
- this merging would be available from the context menu you get if you select multiple entries. Merging candidates are only taken from the entries the user selected.
- This merging functionality could find login entries with same userid+password combination, then choose the oldest entry as "merged" entry. Then merge all URLs from the other entries to this entry, the same with any custom fields and sections with different values (don't add custom fields with same title+value multiple times). And merge the password history of course. Also skip merging for history entries with identical password+timestamp combinations. After merging, add the tag "merged " to the merged entry (so the user can find all processed entries and can check if all is ok). Then delete the items their content was merged into the merged item.

It's an investment in getting new users. I bet the turnover rate from ex-Lastpass users would be higher if you had such a functionality.
When I was new, I merged entries from Password Safe and Chrome. I almost gave up, because it was so tedious - after all, Chrome autofill wasn't that bad, and Password Safe for just a local vault also wasn't that bad for all non-browser login items. A month after the trial expired I gave it a second shot and decided to do the tedious work.

Consider this yet another vote for deduping. My duplicates are somehow the result of switching from a local vault to an online account so I really feel the onus is on Agilebits to fix this.

@kjdiehl

Please don't post the same comment in multiple threads. I've replied to you here: https://1password.community/discussion/comment/670977/#Comment_670977

Let's continue the discussion in the other thread. 🙂

-Dave

I just switched my family over from LastPass and am running into the same trouble. LastPass had some sort of convoluted feature that was called something like "consider as the same website," (I can't see it anymore because I just nuked my account after migrating here,) and it just linked subtly different addresses, external to an individual login entry- basically the same as 1Password's feature allowing a single logon to contain multiple websites. (I think your approach is simpler and makes more sense, but it needs to be automated.) LastPass however was able to do it automatically. I didn't even know their feature existed until I stumbled on it in their settings one day. Because I didn't need to know about it because it made my experience just work.

It's a pain for me, who is tech savvy, to go through ALL my Chase Bank logins for example, (https://secure01a.chase.com/web/auth/logonbox, https://secure01b.chase.com/web/auth/, https://secure03b.chase.com/web/auth/, https://secure05a.chase.com/web/auth/, https://secure05b.chase.com/, https://secure06ea.chase.com/web/auth/logonbox, etc...) and manually perform a SEVEN step process to move EACH website to just one Chase login entry. I have precisely 50 Chase login entries in 1Password at this time. That's 350 clicks just for ONE incredibly common and mainstream website. To ask my 73 year old mother to do this is utterly impossible. It's also far from easy for her to understand that if the first Chase login offered to her by 1Password doesn't work, that another, of maybe dozens, will likely work.

Forgive me if I sound a bit snarky. I just read a dozen separate threads in this forum going back 9 YEARS with many responses from your team disingenuously suggesting that this is an edge case that almost no one encounters and therefore not likely to be prioritized. Heck, so many major websites like Facebook have many different prefixes, as in www.facebook.com vs m.facebook.com, etc. This is a BASIC component of how websites function, and has been for decades. It's absurd that 1Password would consider each of those a separate login and provide no way for users to combine them swiftly, let alone handle it automatically in the background. At the very least, it should be supremely simple to add a manual user-triggered tool that allows me to select all my Chase entries at once, right-click, and then select Merge Websites Into One Login Entry. If there's any conflicting usernames, passwords, or more, just dump it all into the Notes field as appended data.

Please prioritize this entire problem.