Lost Items sometimes [Issue caused by using Synology's cloudstation clients, which was updated]

lutz

Hey Guys,
i have a big Problem with my 1Password installation on two Windows PCs.
I sync the 1password folder via Synology Cloudstation.

Often i miss a single login item, or another item like a server item.
I've checked my Backups an can see, that there are sometimes lost some entrys.
(see the attachment)

I have this lost entrys on both PC's.

Do you have any idea why entrys are lost? How can we catch this error?
I've NEVER delete any item in the last year.

Is there any Diff Tool, to check what kind of items are missed in my current vault?
If i loose all my data in my vault i am homeless programmer ;)

---

1Password Version: 4.6.0.585
Extension Version: Not Provided
OS Version: Win 10
Sync Type: Synology CloudStation

AlexHoffmann

Hi @lutz,

This is odd, to say the least.
I would wager that these are sync conflict resolution issues caused by the Synology server.

I'd like you to write us to support+windows@agilebits.com, referencing this thread, so we can start troubleshooting the issue.

ref: PZH-52723-486

lutz

Hi, your Support hasnt a final solution.
If i activate the Sync over Synology Cloudstation again - some days later items lost again :(

Are you still investigate this issue or?

MikeT

Hi @lutz,

That's because it's not our issue, it's an issue with Synology's client. We cannot fix this by ourselves, Synology has to fix this for you.

Have you updated all of your DSM clients and NAS?

We have not reproduced this issue after installing all of the updates a few months ago, so this should not be an issue anymore as long as your time and date are in sync between all of your computers, NAS, and so on.

If you're still seeing this, then you'll have to get in touch with Synology's support team to figure this out.

lutz

Have you updated all of your DSM clients and NAS?

Yes - all is up2date

We have not reproduced this issue after installing all of the updates a few months ago

I have lost this week more than 20 Items on my Vault
To lost a Item is the worst worst case - i store more then 400 Items in my Vault

What do you think? It is a good idea to switch Back to the agilekeychain format?
There was stored for every item a single file on filesystem - not multiple item in one file or?

AGAlumB

@lutz: I'm sorry to hear that. It sounds like AgileKeychain's per-item files may be a better fit for how your NAS works, as opposed to the bands used by OPVault. Were you having better results with the previous vault format?

lutz

Is there any chance to export the dir-structure?
If i Export all objects from opvault and import to agilekeychain all dirs missing

AlexHoffmann

Sorry @lutz that is not possible at the moment. We are very sorry for the inconvenience.

And to answer your earlier question: we are keeping an eye on this.

AlexHoffmann

To clarify: This is a bug that we're investigating right now, related to exporting from the OPVault format while it works fine with Agilekeychain format. We're hoping to have a fix soon.

ref: OPW-525

lutz

@AlexHoffmann Sorry, but which bug do you try to fix?

• The Export from OPVaiut to Keychain
• • The Sync issue with missing items?

MikeT

Hi @lutz,

The missing folder structure from the OPVault-created 1PIF. Once we fix this bug, you will then be able to export again and import into agilekeychain with folders intact.

The Synology Cloudstation issue isn't something we can fix at the moment.

lutz

I have new test informations for you.

I've created a new Test-Szenario in my Lab.

One Synology with Cloudstation and ONLY ONE Windows 10 PC with 1 Password and Cloud Station to Sync the Vault Folder.
The Problem cannot be a file conflict - inside of the Synology.

If the WindowsPC run, i get after some days the error.
Synology Sync Tool, cant find (or think that cant find) some index_X.js files from the vault, and remove them from the remote site.

To recreate the issue a single PC is enough.

Is there any moment (eg. store a new password, create a backup etc.) where 1 password block the index files on the filesystem?
I think it can be a false positive interpretation from the cloud station.

Today my single Client has removed 2 indexfiles after add some passwords to my vault.

What do you think about it?

MikeT

Hi @lutz,

Thanks for the additional information, can you provide screenshots from Synology about these files and the removal notices, including its timestamps as well? There should not be any removal in the process at all.

If Synology thinks the file is missing, it should sync the copy it has to the other side, not remove it, which is the incorrect behavior.

1Password doesn't lock the files during its performance, otherwise 1Password Helper will not work since it works on the same files that the main program works on. In addition, we'd see this same issue with any other sync services, we didn't.

lutz

Yes i will collect some screenshots and logfiles

If Synology thinks the file is missing, it should sync the copy it has to the other side, not remove it, which is the incorrect behavior.

No, if a file local is missing - it is removed on the "cloud-side". Thos behaviour is correct.
Dropbox and other work in a same way. If i delete a file in my local copy, it will be removed in the remote copy

lutz

Hi,

i've found these informations from the case yesterday.

In the first screenshot you can see that 2 index files are removed from the remote site (client think the files are localy deleted)

You cann see that i've restored the files from 1password backup

In the timeslot where the client has removed, the computer was locked. (WIN+L)
You can see this information in the Windows logon logfile

So the issue can NOT be a user-interaction like store new passes or edit some stuff

I've opened a Synology Ticket too #711170

MikeT

Hi @lutz,

Wow, I would be very interested in hearing what they say for the reason of any deletions that are occurring while the system is locked.

Did you see any other files being deleted or is it always consistently the band files?

No, if a file local is missing - it is removed on the "cloud-side". Thos behaviour is correct. Dropbox and other work in a same way. If i delete a file in my local copy, it will be removed in the remote copy Dropbox and other work in a same way. If i delete a file in my local copy, it will be removed in the remote copy

Let me put it the other way around, how do you know the difference between a new file created in the remote cloud and the local file deleted on the local side? Who wins in this situation?

If the file is newer on the remote cloud and Synology has no history of the file existing on the local side, it must copy it from the remote. If it has a history of this file intentionally deleted and the remote cloud has an older timestamp, then it must remove it from the cloud. However, what happens if the database is wrong on one side or has the wrong timestamp recorded?

1Password does not remove files, it empties the content and leave behind a shell as tombstones to prevent other sides into thinking that the files are missing. In other words, it does not remove files and thus Synology should not be removing in any situations.

What I also recommend for you to do is monitor the folder directly, look and see if the files are actually removed.

lutz

Did you see any other files being deleted or is it always consistently the band files?

Only the 1Password JS Index files are removed!

I have many more files like big OneNote Books, Pictures etc. on the CloudStation.
The Problem are only the Index files from 1Password.

OneNote Sync works perfect over the Synology solution.

how do you know the difference between a new file created in the remote cloud and the local file deleted on the local side? Who wins in this situation?

• new file created on cloud side :: client download this file as new local file
• local file will be deleted :: client will remove the file on remote side

Who wins in this situation?

If you have a duplicate filename conflicht - Synology renamed one of the files with "-conflict."

• I've never seen conflict files like these in my vault or Synology log.

What I also recommend for you to do is monitor the folder directly, look and see if the files are actually removed.

Do you have a easy solution without to scripting itself?

MikeT

Hi @lutz,

Try ProcMon: https://technet.microsoft.com/en-us/sysinternals/processmonitor.aspx

lutz

sweet! I've setup a capture to the vault dir

MikeT

Please keep us informed and probably Synology as well if you find anything in the capture.

lutz

at the moment i see only the explorer.exe and 1password.exe - no Synology :-P

Okay, i will create a trace with catched error.
Thanks for exzellent technical help!

MikeT

On behalf of the team, you're welcome.

lutz

1) I have recyle 34 items on my 1password bin.
2) Programm runs and hang for two seconds
3) I can see that the counter (on left sitebar behind logins) go down
4) Create a new Backup
4.1) Before cleanup bin my vault has 534 items - after 496 items
-> There are some lost :-D

I have a logfile: "Logfile-after-clean-bin.PML"
-> New Logfile started

After the above Problem
1) I have restored my Vault from a Backup with 534 items
2) delete the bin again
2.1) all is ok
3) Wait some minutes (the i will see much traffic on process monitor)
3.1) I have Lost again some Items in my vault
3.2) 1Password fragged totally :-P (See my Screenshot)
3.3) If i create a new backup from my opvault vault, it will create a agilekeychain backup with 0 Items?

I have a logfile: "Logfile-crashed-vault-after-restore-from-backup.PML"
-> New Logfile started

3.4) "Reapir Vault" Function has no effect - backups are empty and agilekechain backups.
4) Restore Vault from Backup with 534 Items again (like step 1)
5) Wait some Minutes - all looks fine
6) Wait some Minutes - no items lost / no files remove from synology-sync
7) Create a new Backup
7.1) Backup are correct created with 534 Items

If the error is here, it seems to be every time the same files which are removed?
-> band_f / band_8 (no clue, if its help?)

Is there any secure way, how i can send the Logfiles from process monitor?
I dont like to upload these files to the forum :-}

lutz

I've send the Logfiles with a reference to my last post to your support email address via support+windows@agilebits.com

MikeT

Hi @lutz,

Thanks, we have the email and will analyze it as soon as possible.

ref: KAS-42115-575