Inconsistent item counts on backup/export/import

Hatclub

I'm finally getting around to trying to switch to opvault; however the following does not exactly inspire confidence:

Item counts in old agile vault total 729
Backup - filename suggests backup contains 726 items
Exporting to 1pif - "you have exported 912 items"
Importing from 1pif - "896 items imported" (plus a warning about overwriting a duplicate, despite no visible duplicate existing in agile vault)
Item counts in opvault total 714 (logins down by 1, generated passwords down by 11)
Exporting from opvault to 1pif - "you have exported 870 items"

?!?

---

1Password Version: 4.6.0.604
Extension Version: n/a
OS Version: Windows 10 x64 1607
Sync Type: Dropbox
Referrer: forum-search:export item count

rjh

Hi @Hatclub , Sorry that you're getting inconsistent item counts when synchronising 1Password.
Have you by any chance created a Personal Vault within 1Password, alongside your Primary Vault?
This might explain the discrepancy. I don't have Windows, but you should see an option (top left of the 1Password window) "All Vaults"
If you open the menu, are there multiple vaults?

Please get back to us on this, so that we can help further.

Cheers,
Rob

● Disclaimer: Please note that I am not a member of the AgileBits team.

Hatclub

Thanks for the reply @rjh

I don't believe 1Password for windows supports multiple concurrent vaults (there is quite a gulf in functionality/behaviour between the mac and windows versions at present) - you can only switch between vaults (and I only have the one!), so it doesn't look likely that this is the cause here! :(

rjh

Hi again @Hatclub ,

I just found a screenshot that displays the window that I mentioned.
If you click on the 'button' (labeled 'Personal' here), do you see local vaults alongside your Primary Vault?

Cheers,
Rob

Hatclub

Hi @rjh

That's the Beta version 6 (or the windows 10 "app" beta, I'm not sure which).

I'm using 4.6 (the current stable windows build) here. There is no equivalent functionality or appearance in the stable windows build.

Phil

MikeT

Hi @Hatclub,

The reason you're seeing different item counts are because they can include folders and other items that are not included in the item count for the backups. So, backups would say 66 but if you have 3 folders, the export will say 69 instead.

Also, don't forget to count the items in your Trash, these are counted for the export as well.

There are some issues with exporting folders via OPVault that we're working on right now, it won't include them.

(plus a warning about overwriting a duplicate, despite no visible duplicate existing in agile vault)

It's based on the same type and title. Did you tell it to not overwrite it? If you have two Login items with the same title like Amazon, that is counted as a duplicate for the importer and it will overwrite the older item with the newer one if you said yes and this can reduce the item count.

In other words, when importing 1PIF file, say yes to all to import the items but say no to all when it ask if you'd like to overwrite anything.

1PIF was originally used to update databases between different computers, prior to syncing, so that's why you see this option. The next version won't do this anymore.

Hatclub

Hi @MikeT

Thanks for the info!

The reason you're seeing different item counts are because they can include folders and other items that are not included in the item count for the backups. So, backups would say 66 but if you have 3 folders, the export will say 69 instead.

Okay. I have 170 items in trash, 13 folders - added to my previous item count of 729, this means that the export count of 912 would be correct.

I have checked again, and a fresh backup I just did reflects the current 730 item count, so it seems possible my backup was not "bang up to date" at the time I did the export (it is possible I added an item between backup and export - I generate a lot of credentials in my line of work).

Also, don't forget to count the items in your Trash, these are counted for the export as well.

Okay - and the trash is where my dupes are hiding, too. I had previously searched "All" but obviously trash items (which I'd forgotten about) aren't shown there.

Let's go again: My export count is now 916 - but 730 items, 13 folders, 170 deleted only comes to 913, so I seem to have 3 "extra" items I can't account for in my export.

I create a new opvault, tell it not to overwrite anything, I now get 900 items imported. This would make sense if we take 13 folders from my 913 expected item count, but then my mystery 3 items have vanished again.

My item count in my opvault now totals 730, as it should.

In summary - more feedback:

• It might be useful for the trash folder to have a count against it to make it more obvious when looking at the interface that it is "another place" where unique items are stored.

• The export item count appears to be off by three somehow. I have just realised I have 3 favourites - are the favourite markers counted as an item on export (but again, not on import)?

• It seems possible that on my last attempt to import, because I told it to overwrite dupes (chalk that one up to me not reading the "converting" guide all the way to the end after I was bamboozled by the exported item count being unexpectedly high), it removed something from my non-trash section in order to keep a dupe in the Trash, which would have resulted in information loss in the new opvault (not that I was going to delete the agilevault from my cold storage backups any time soon). The takeaway here is that 1PIF imports should definitely not offer an overwrite option given there is no way to know when overwriting which item is newer and which item is older, so it's good that this is going to disappear.

• When importing, it would be great if folders were counted on their way in, if they're going to be counted on their way out (or, stop counting them on the way out, or better yet, present folders separately in the count - "730 items, 170 trash items, 13 folders [exported|imported]" would be the clearest information to present to the end user). There's really no margin for error when it comes to handling this kind of information, so inconsistency in how items are counted is always going to ring alarm bells.

MikeT

Hi @Hatclub,

One more thing, the item counts are not real-time, we only update it when you click on the category. The main reason is that 1Password 4 reads your 1Password vault directly, the vault has a single file per item for the Agile Keychain and 16 band files for OPVault.

So, to use the least amount of the system's resources and to maintain performance, we only read when we need to. In this case, when you click on the category, we read all of your files to find the data you need. That's why we don't have the item counts on all of the sidebar items, to avoid doing any unneeded work.

In the next version of 1Password for Windows that we're working on, all items will be stored in a single database structure that we can run queries on, which mean item counts can be done fairly quick. In fact, in the latest 1Password 6 Beta for Windows, it will include a proper item count on top like so:

It might be useful for the trash folder to have a count against it to make it more obvious when looking at the interface that it is "another place" where unique items are stored.

Agreed, although I'm not sure we'll be able to do this for 1Password 4 but I'll ask.

By the way, already done in 1Password 6 Beta:

The export item count appears to be off by three somehow. I have just realised I have 3 favourites - are the favourite markers counted as an item on export (but again, not on import)?

No, since the item's favorite status is stored in the item itself.

Did you include any nested folders? Nested folders are separate items as well.

When importing, it would be great if folders were counted on their way in, if they're going to be counted on their way out (or, stop counting them on the way out, or better yet, present folders separately in the count - "730 items, 170 trash items, 13 folders [exported|imported]" would be the clearest information to present to the end user). There's really no margin for error when it comes to handling this kind of information, so inconsistency in how items are counted is always going to ring alarm bells.

I agree, we're doing a major rethinking of this in 1Password 6. It will have an import process that looks something like this:

This is from our macOS version of 1Password that we will be modeling for 1Password 6 for Windows. We will also list any information about which items couldn't be imported due to malformations.