Tool for detecting and removing duplicates? [Available in 8.10.18]



  • ronread
    Community Member

    I also miss the previous 'find duplicate' tool in V7. After finally upgrading after figuring out I had to first import my standalone vault syncing with dropbox (AB: you need to make this requirement MORE prominent on your website!), I had several dups, sometimes 3 copies for each item. Funny thing was, some of the dups did not have the attachments that were included in the item. I finally manually cleaned out dups (down to around 700 items), and in some cases had to reattach files that landed in the "Salvage" vault. I think I've got things under control, but the upgrade was not smooth. So back to the original point: Is the loss of finding dups due to the switch to Electron?

  • Hi @CFS, I'm sorry to hear of this experience. Our Mac specialist team would be happy to help with this - please send us a brief message at [email protected], with a link to your post (so that we know it's you, and you don't have to tell the whole story again), and we'll get you some customized help.

  • PeterG_1P
    edited May 2022

    @ronread, I'm sorry to learn you encountered that duplicate issue. If there's anything further we can do to help with your specific case, let me know and I'll be happy to do what I can.

    Meanwhile, we do have a request for a Duplicate Detective 🕵️‍♂️ feature (I'm lobbying hard for that terminology, as a fan of alliteration), and are collecting customer votes for it to get a sense of the demand. I can really see why it'd be useful in your case and am happy to add a vote on your behalf.

    To your question:

    So back to the original point: Is the loss of finding dups due to the switch to Electron?

    No, to my knowledge Electron didn't have anything to do with this. One of the big challenges of creating 1Password 8 is that it's an entirely new app - a true generational shift - and not an iteration built upon 1Password 7. So that meant "coding from the ground up", as it were.

    There are a lot of great things about that (the memory security that Rust offers, much faster ability to roll out bug fixes consistently cross-platform, Universal Autofill, etc). But we do have some features that were in some versions of 1Password 7 (which wasn't nearly as consistent from OS to OS - Windows never had a duplicate removal feature) that aren't in the current release. This is certainly one I'd like you to be able to use, and I'll advocate for it on your behalf.

    ref: IDEA-I-615

    note: edited for atrocious grammar

  • ronread
    Community Member

    Thanks! I look forward to it, though I've only ever had duplicate problems in this upgrade. A feature I'd prioritize over that is the ability to have "local vaults" (for security requirements of clients) made by the per-vault option to sync or not to sync.

  • elxris
    Community Member

    I migrated from v7 from Apple Store, to v8 from your website, and I also find this duplicate issue very annoying. I feel like the culprit here is the migration tool. I feel like it imported some old iCloud backup, that I no longer use since I am on subscription. I have hundreds of duplicates now. Dumb tool.

    Also, while trying to delete duplicates, if you press delete key and then down arrow to select the next one the selection goes to the first element of the list and I need to scroll all the way back to where I was. Very annoying as I need to use keyboard + mouse to properly delete the duplicates.

    This version is no longer convenient to use as I never know which one of the credentials I must use.

    While I understand this is a new codebase, I feel like the migration processes are less polished and we are featureless. I am sorry to say I feel disappointed. I remember being here and proud with 1Password since 2015.

  • PeterG_1P
    edited May 2022

    Hi @elxris, I'm sorry to learn of your experience with this. We do appreciate having your feedback on the migration process.

    Since it sounds like you still have a lot of those duplicates at the moment, I invite you to send a brief email to [email protected], with a link to your post here. That way our support team can quickly catch up with your current situation and help you (literally) sort it out. And again, my apologies for the inconvenience you've experienced so far. We'd love to help set it right.

  • cornelln
    Community Member
    edited May 2022

    Several of the people here are in this situation due to some issue occurring on migration from 1Password 7 to 1Password 8. This issue on its own is a failure of a critical part of the migration function and seems unaddressed in the thread. The system quite simply be checking much better than it does to avoid that fate and recover from it.

    The fact that manual duplicate clean up is the solution on a database that for many can approach 1k or greater is simply not good enough. If migration issues occur the duplicate search should really be moved up your priority queue.

    Lastly I ended up just deleting everything from the account online and then re-importing. This would be my suggestion to anyone who identifies this problem before using the new data base too much.

    This issue alone mostly just got me thinking again about bailing on the product all together. After all it's simply database app at it's core function and if it can't be trusted w that what's the use?

    In general the migration process didn't feel well designed to me. For example why would the system not remind the user to go ahead and detach the other vault? This upgrade process is not good, keep working on that or just expect additional users in here over time complaining about that (well the one's that bother to give feedback).
    For example presently on iOS I have 2 Vaults in iOS 7. This is how the UI is confusing

    All Vaults Screen
    1. "Vault for Saving" was still set to iCloud... Not updated to save to the online account... So right there if I didn't change that I would have the issue of saving newly created passwords to a vault I am migrating off of.

    1. The old vault was still "On" so of course you end up w a confusing set of duplicates that way too

    2. The naming structure is just confusing to someone not working for the company.

    "Vaults included in all vaults"
    You have
    [FirstName LastName] ON
    Personal ON

    On iCloud
    Primary ON

    Let me ask you do you think that the average non technical user can tell you quickly which vault is which w this kind of naming structure? Or that one is their old vault from 7 vs where they are trying to migrate too?

    In short your UI and migration does the user very little favors to be honest to the point its borderline user hostile. What % of users hose their migration 10-20%?
    Another issue you have a promotional upsell function where users can get a discounted price - that's fine... But does this disappear after you pay and sign in with an account - no it doesn't. So is there no QA validation of something as simple as 'if user has paid account stop displaying the promotional notice'... Guess not? Seems like a simple nuance but it's not accounted for.

  • lyricnz2
    Community Member
    edited June 2022

    Yes, this lack of function is unfortunate!
    If you have decent password hygiene, you might be able to find the duplicates easier in the Watchtower->Duplicate Passwords view.

  • jordazzle
    Community Member

    @Chris Messina thanks for all your posts and bringing this to light. I'm in the exact same boat as you. I used the standalone version of 1password for many many years. And finally decided... I get enough value out of this that I don't mind paying a small monthly fee for the one software application that I use every single day, all day long.

    When we made the move to the new cloud subscription service everything seemed okay. But now as I'm going to use a login identity I'm seeing at least two versions of the exact same credentials.

    I recognize this may take some significant development to build a duplicate item cleanup tool, but I think it would really helpful to streamline our online identities.

  • Chris Messina
    Chris Messina
    Community Member

    Yep — and it's gotten worse as I update passwords... now I have TWO entries and especially when they sync to my phone, it's nearly IMPOSSIBLE to tell which is the correct/current credential! 😣

  • Hi folks, thanks for your feedback here. This is certainly an issue I'm happy to advocate for on your behalf.

    To @spinoza's point: 1Password has grown incredibly fast in the last few years, and we've re-organized and re-thought the way we prioritize and triage issues along the way. What I mean by this is: just because something was requested before and hasn't happened, doesn't mean it's off the table. While I don't have anything specific to share on this feature request at the moment, I can tell you honestly that we in customer service are tallying the votes and that community demand does matter.

    So if this is something you feel would be useful for you, and you haven't let us know already, I welcome you to do so. I'll advocate for your use case in any way I can.

    Speaking of which, all the votes from this thread which were not previously added have now been tallied (and we already had votes for @Chris Messina, @spinoza, and @cornelln). 👍

  • CFS
    Community Member

    Since I don't have access to my Mac computer -- and won't for several months -- I finally resolved this duplicates problem by spending four hours on 1Password 8 on my Windows PC, going through about 1,000 entries and archiving the invalid duplicates. (Which, as I explained earlier, weren't really duplicates, just earlier versions of the correct entry.) A delightful way to pass a Sunday afternoon. :(

  • czyzczyz
    Community Member

    Please add a vote for me. In the migration from 1password 7 to v8 I've ended up with thousands of duplicates. I've been using 1password since February of 2008 and have never run into this kind of frustration.

    I spent a half hour manually command-clicking and deleting duplicates and barely made a dent in it. The duplicates crowd my view of passwords whenever I try to use 1password to login to a site.

    The duplicates are all exact dupes, with all entries and content and the same modification and creation timestamps. There wouldn't be a need for a tool that does fancy comparisons between records -- just quickly scan through all of them and generate a hash for each, then delete one of each matching pair. Maybe I'll see if I can figure out a way to automate this with 1password-cli.

  • Hi @olivierP and @czyzczyz, I have added votes on the issue for each of you in turn. Thank you for taking the time to let us know why you find value in that feature and the specific circumstances you've run into. I'm passing it on to our development team now and appreciate your sharing this with us - and the thoughts on how a solution could be implemented.

    ref: IDEA-I-615

  • Hi @ctphoto, I don't have any updates to supply at the moment but can tell you that I'm happy to pass on your feedback - and have now done so. Thanks for letting us know this is a priority for you. 🙏

  • mikesilva
    Community Member

    +1 on this issue for me.

  • 👍 ✏️

  • kinthecity
    Community Member

    @1Password team, please aggressively move this up your queue... In today's day and age, this is a fairly "simple" ask / obvious, table-stakes type of function that really puts a significant damper on your paid user experience. I did a fair amount of research before deciding to move forward with 1Pw, and I am actually shocked that this isn't showing up as a significant downside in your online reviews and by media outlets... It's only a matter of time, and it's going to continue to significantly negatively affect ux until it's fixed. The fact that this has been deprioritized for 18 months is... disappointing, to say the least.

  • martinroy
    Community Member

    @1Password, you should put this in the Watchtower. You cannot have two login with the same username.

    As a new user, we need to import several password database into 1Password and we need a tool to check if there is duplicate.

    Martin Roy

  • Hi folks, I'm sorry we don't any any further news to share at this time regarding this feature, but I've added these new votes and comments for the teams review. Thanks for the feedback.

  • mikefoley
    Community Member

    I'm running into this issue as well. On a Mac, running 8. Also running 8 on Windows.

This discussion has been closed.