Tool for detecting and removing duplicates? [Available in 8.10.18]

Unknown

This content has been removed.

andrew.l_1P

Hi @et01267,

Thanks for your feedback and welcome to 1Password!

I don't have any updates regarding a duplicate tool at the moment, but if you're seeing duplicate items from a repeat import on an otherwise empty account and you still have the source file, I recommend creating a new vault and importing your data again.

If your duplicates are in a user-made vault you can replace that vault with the new one. If your duplicates are in your Private vault you can delete its contents and then move the entirety of the new vault into your Private vault.

I recognize this is only viable for new accounts where re-importing is an option though so I'll add your feedback and use case to our internal discussion.

Let me know if you have any questions. 🙂

ref: IDEA-I-615

vunsal

In my case, I have noticed that the duplication is actually in the exported LastPass passwords file. I went through that file line by line to remove the duplications before uploading to 1Password.

Unknown

This content has been removed.

RowanFrancis

@andrew.l_1P i have just opened a 1p account as i am considering moving from lastp... (unmentionable) i have a family account and i moved everything across and i have 3 duplicates for every entry , so over 4000 entires in my single vault , I have yet to load the data for the rest of the families accounts in light of this.
we are now in 2023 and i see the lack of a duplicate tool has been hanging around here for over 8 months now like a bad smell, is there any light at the end of these requests for a duplicate tool or should i just move on ?

andrew.l_1P

Hi @RowanFrancis,

Thanks for your consideration, I'm sorry to hear your initial import didn't go as smoothly as we would hope. You can find our latest information about this feature request in this post: https://1password.community/discussion/comment/662923/#Comment_662923 but essentially we're focusing our efforts on understanding and preventing duplicates in the first place.

If you're open to sharing more about your import workflow, I'm happy to help troubleshoot.

CFS

As I indicated in May 2022, the cleanup of my many duplicates took hours. Now I see, from @RowanFrancis, that helping customers solve this problem isn't a priority for 1Password. Which means, I assume, that they won't be doing anything about it at all. Meanwhile I'm now getting notifications urging me to update my 1Password app for Android to version 8. Not a chance in hell. I'll limp along with what I have now for as long as possible.,

Unknown

This content has been removed.

1thierry

Hello,
I understood it is not coming into new features...
I still needed to add another message to this thread, with my problem, similar to others when upgrading / mergin... with tons of duplicates.
I wish there was better solution, to avoid going one by one...

kjdiehl

I just switched my family over from LastPass and am running into the same trouble. LastPass had some sort of convoluted feature that was called something like "consider as the same website," (I can't see it anymore because I just nuked my account after migrating here,) and it just linked subtly different addresses, external to an individual login entry- basically the same as 1Password's feature allowing a single logon to contain multiple websites. (I think your approach is simpler and makes more sense, but it needs to be automated.) LastPass however was able to do it automatically. I didn't even know their feature existed until I stumbled on it in their settings one day. Because I didn't need to know about it because it made my experience just work.

It's a pain for me, who is tech savvy, to go through ALL my Chase Bank logins for example, (https://secure01a.chase.com/web/auth/logonbox, https://secure01b.chase.com/web/auth/, https://secure03b.chase.com/web/auth/, https://secure05a.chase.com/web/auth/, https://secure05b.chase.com/, https://secure06ea.chase.com/web/auth/logonbox, etc...) and manually perform a SEVEN step process to move EACH website to just one Chase login entry. I have precisely 50 Chase login entries in 1Password at this time. That's 350 clicks just for ONE incredibly common and mainstream website. To ask my 73 year old mother to do this is utterly impossible. It's also far from easy for her to understand that if the first Chase login offered to her by 1Password doesn't work, that another, of maybe dozens, will likely work.

Forgive me if I sound a bit snarky. I just read a dozen separate threads in this forum going back 9 YEARS with many responses from your team disingenuously suggesting that this is an edge case that almost no one encounters and therefore not likely to be prioritized. Heck, so many major websites like Facebook have many different prefixes, as in www.facebook.com vs m.facebook.com, etc. This is a BASIC component of how websites function, and has been for decades. It's absurd that 1Password would consider each of those a separate login and provide no way for users to combine them swiftly, let alone handle it automatically in the background. At the very least, it should be supremely simple to add a manual user-triggered tool that allows me to select all my Chase entries at once, right-click, and then select Merge Websites Into One Login Entry. If there's any conflicting usernames, passwords, or more, just dump it all into the Notes field as appended data.

Please prioritize this entire problem.

Dave_1P

@kjdiehl

Please don't post the same comment in multiple threads. I've replied to you here: https://1password.community/discussion/comment/670977/#Comment_670977

Let's continue the discussion in the other thread. 🙂

-Dave

Peregrine

Consider this yet another vote for deduping. My duplicates are somehow the result of switching from a local vault to an online account so I really feel the onus is on Agilebits to fix this.

Tertius3

In an ideal world, nobody will duplicate data at import time, but actually it will happen. I will not discuss how it can happen or can be avoided, only that it will happen - proof is this thread and others. Duplicate items are not always user error. Switching to a decent password manager often means merging from multiple sources, then consolidating: import from previous password manager plus Chrome's plus Firefox's plus Safari's internal password managers.

One does import once directly after account creation, then never again. You at Agilebits do it never again, so you don't see really any need for postprocessing aid. However, for a new user it's crucial, it's a barrier.

With duplicate items I mean login items that contain the same userid+password combination.

What I propose you can do about this:

• give an overview how many items match above description, in a similar way Watchtower does this. Also create filters, so the user can see a list of only matching entries.
• also give an overview how many items match with url+userid, but different password. These might represent current and obsolete entries of the same login. The category could be something like "unable to merge", "merge conflict" or "password mismatch". Also make a filter for these.
• explain if duplicates are distributed over multiple vaults. If 2 vaults contain the same, because the user imported the same 2 times, the user should see this and the easy fix for the user is to just delete one of these vaults
• make it clear over how many different vaults how many duplicated entries are distributed
• add the opportunity to "merge" duplicate entries in one vault, in spirit similar to functions to merge different address entries in address books in other apps. Merge only items within the same vault.
• this merging would be available from the context menu you get if you select multiple entries. Merging candidates are only taken from the entries the user selected.
• This merging functionality could find login entries with same userid+password combination, then choose the oldest entry as "merged" entry. Then merge all URLs from the other entries to this entry, the same with any custom fields and sections with different values (don't add custom fields with same title+value multiple times). And merge the password history of course. Also skip merging for history entries with identical password+timestamp combinations. After merging, add the tag "merged " to the merged entry (so the user can find all processed entries and can check if all is ok). Then delete the items their content was merged into the merged item.

It's an investment in getting new users. I bet the turnover rate from ex-Lastpass users would be higher if you had such a functionality.
When I was new, I merged entries from Password Safe and Chrome. I almost gave up, because it was so tedious - after all, Chrome autofill wasn't that bad, and Password Safe for just a local vault also wasn't that bad for all non-browser login items. A month after the trial expired I gave it a second shot and decided to do the tedious work.

PeterG_1P

Hi @peregrine, sorry that this happened in the switch. I have added a vote for the feature on your behalf. Thanks for letting us know why this matters for you.

PeterG_1P

Thank you @Tertius3 for the in-depth and considered post here. Your reasoning makes sense to me. I'll pass this on to our Product team immediately so that it can inform the process going forward, and of course will update here should we have more to share. Many thanks 🙏

siberian

+1

austenlennon

I am looking for a password manager app... but the lack of facility to remove duplicates is a non- starter... I'll keep looking.

Unknown

This content has been removed.

Unknown

This content has been removed.

achiles

2023 and we are still counting votes... Sorry fellows, we have been forgotten :(

marcosuave

Product development work for a duplicate management tool aside, it would be super helpful to have a detailed explanation somewhere from you all as to why and how duplicates end up in our accounts. I, for one, have so many duplicate logins and they seem to be replicating. I even have what appears to be quadruplicates for some login items. There are many ways to tackle this widely-experienced issue with 1password, developing new tooling to manage dupes is just one way. Behavior change on the part of us users is also a way to approach this, but we need a good understanding of why the heck all these duplicates end up in our accounts in order to stop the bleeding.

Unknown

This content has been removed.

mayone

+1
updated from 7 to 8 on a macbook and then dozen of duplicates appeared...

Ben

Hi @mayone

Are you seeing these duplicates in 1Password 8? If so, are the duplicate items in the same vault as each other, or for example is Duplicate A in Vault 1 and Duplicate B is in Vault 2? Please let me know. We'd like to help resolve this problem for you.

Ben

mayone

Hi Ben.
Yes, I see the duplicates in 1Password 8 after upgrade from 7 on one of my macs. I use only one vault, very simple configuration.
Thank you.

Ben

@mayone

Gotcha. Thank you. You mentioned this is happening on one of your Macs. Do you have other devices with 1Password 8 where you are not seeing the duplicates? If so, that would indicate to me that there is some sort of configuration issue on the one Mac. My first thought would be that you may've ended up with multiple 1Password accounts and are signed into both on that one.

If you use your web browser to access our web app at https://start.1password.com do you see the duplication there? If you do not, then let's check 1Password on that Mac to see if my theory is correct. :)

To check:

1. Open and unlock 1Password on the impacted Mac
2. In the upper left of the 1Password window select the ⬇️
3. From the resulting menu select 'Manage Accounts'

Are there multiple accounts listed there?

Ben

marcosuave

FWIW, I also use Macs exclusively. I have the app installed on multiple machines, as well as the browser extensions installed (primarily use Chrome, though.) I see the duplicates everywhere: the on-page autofill list next to a field, the list from the browser extension icon, the mac apps, my iOs apps, and on. Oddly, many of the duplicates have one login item saved as item_name (username) and another version with everything the same except the (username). Not sure if that's helpful but I'm hoping so--this is getting harder to work around. I just counted and it appears I have 8 repeats of the same login item. So not just duplicates, but also octuplicates. Is it possible the Chrome browser extension is inadvertently saving logins without asking for confirmation due to a mismatch in something like http/s or the full login URL?

mayone

Hi Ben.
No, it happens an all my mac devices (mac, ipad, iPhone) all with 1Password 8. The duplicates appeared when I’ve updated my last mac from version 7 to 8 and then propagates to other devices. Before upgrading this mac from 7 to 8 everything was ok.

mayone

Dear @Ben I've checked and I have only one account and only one vault. Now all my duplicates are propagated in any devices and in online account.
I renewed my subscription just a few days ago. If I had know, I wouldn't have wasted my money. Paying for bad service like this is really frustrating and meaningless.
Waiting for some (good) news.

Ben

Hi @mayone,

I'm afraid if the duplicates are in the same vault and on all devices there isn't an easy fix here. When you were using 1Password 7 were you using a standalone vault? If so, do you still have that standalone vault? What I suspect may have happened is that your data was migrated twice from standalone to 1Password.com. If you do still have access to it what we could help you through is deleting all of the data in your 1Password.com account and then re-running the migration (once).

To best explore this option I'd like to get you in touch with our migration team. They can be reached via email at support+tradein@1password.com. With your email please include:

1. Your forum username: mayone
2. A link to this thread: https://1password.community/discussion/comment/681889/#Comment_681889
3. A note that you have duplicate items in 1Password after migrating and have already checked that you do not have multiple vaults

We'll do everything we can to help you get to a good state.

Ben