1PW7 can not star/unstar/re-star cleanly + backup zip files choking Dropbox syncing
Accidentally starred an entry without issue. When I tried clicking to unstar the star was not removed. 20 semi-rapid clicks later the star was still starred. I pause, star suddenly unstarred. 10 semi-rapid clicks later, star not re-starred. If I time my clicks like one-one-thousand, two-one-thousand, three-one-thousand (so three clicks about 1 second apart), the star is restarred.
I'm not sure this is related, but while this was all going on, 1Password was creating a zip file in the opvault back up. Dropbox was trying to sync this file but failing because the zip file was locked. Once the zip file creation was complete, Dropbox syncing was choking on the 111mb back up zip file. The file lock persisted for a couple of minutes. Syncing still has not completed after 5 minutes or so. Sync upload was in the 500 kb/sec range. Home internet up speeds 5Mbps. Dropbox does not have an upload speed limit set.
Edit:
So in the above Dropbox note, the file created was 2018-03-24-21-27-08.zip. Less than 5 minutes later, I am getting another back up zip file being created 2018-03-24-21-49-46.zip while the -08.zip had been deleted
C:\Users\myusername\Dropbox\Apps\1Password\1Password.opvault.backup
Is this normal behavior to be zipping back up files every five minutes? That seems excessive to me.
1Password Version: 1Password 7
Extension Version: Not Provided
OS Version: Win8.1
Sync Type: Dropbox
Comments
-
Thanks for reporting this.
That doesn't sound normal, Dropbox shouldn't be locking the file like that. I wonder if it is a sync conflict, it could explain the initial issue you saw where unstarring doesn't work.
1Password only creates an additional backup when you make a change and then it deletes the older backup on that day. If it is doing it every 5 minutes without any changes made by you, you may be experiencing a sync conflict.
Can you copy the newest backup to the desktop, extract it, and open the
defaultsubfolder in the OPvault folder, do you see any "conflict" in any of the files?Can you also send us your 1Password diagnostics report. Please use this guide to generate the report and email it to us at support+windows@agilebits.com. Also, in the email, include the link to this thread along with your forum username, so that we can connect the email to this thread.
Let us know here when you've sent it, so we can confirm we got the email.
0 -
@MikeT I think I managed to catch the issue while it was happening and run a diagnostic report. Hopefully it will show what is happening. I am beginning to suspect that some of the overall 1PW7 application laginess and flakiness is due to this back up file issue. The slow application response while this is happening was enough to drive me back to 1PW4 to do actual work in the application.
I extracted the last zip file to desktop as asked. You mean for me to search for a filename containing the string "conflict"? If that is what you mean, a file explorer search for that name returned ZERO results.
This morning after firing up the laptop after a fresh reboot, this locking issue appeared again. Screen shot in line. Dropbox appears to be trying and succeeding to sync files after the back up file is successfully created and no longer in use by 1PW7. Right now, Dropbox syncing is up to date with no lock files display.
Just submitted this forum link and diagnostic report as requested. Fingers crossed!
0 -
I don't see any diagnostics report from you yet.
Can you take a screenshot of the folder content of the extracted
defaultfolder. I don't trust Windows File Explorer as it almost never finds anything I need.0 -
@MikeT Okay, think I got what you what. First screenshot is what the Dropbox directory has. Second is the extracted zip file contents on the Desktop. Let me know if you need anything further. I sent the diagnostic report in just a minute or so before my post to you. Hopefully you should see it now.
Right now, I have 1PW7 closed and 1PW4 open. Dropbox is fully synced and 1PW4 is behaving normally.
Ah P.S. I gave only the first screen of files in the attached pic. Let me know if you want everything.
0 -
Sort it by name and only focus on these files:
Any files like:
band)_1 (conflict copy).js?0 -
@MikeT Here you go. Looks normal to me, without any conflict copies.
I don't have a screen print of it, but previously when I hung my cursor over the Dropbox icon in the icon tray and saw the back up zip file lock message, I have seen Dropbox syncing these band_X.js files while the lock on the zip file was happening..
0 -
Ok, it's not a sync conflict.
The problem may be storing the backups into the sync folder where your original vault is. We're working on moving it to its own dedicated folder within your 1Password folder. This would avoid having Dropbox upload your huge vault backup files in the first place.
0 -
@MikeT Thanks. Still doesn't explain why many back up zip files are being created. 1PW7 performance, non-responsivity to clicks and refresh after up/down arrow navigation, seems to be significantly affected while the the backups are happening. Unfortunately since there is no back up configuration information in 1PW7 preferences, I can't confirm details myself.
0 -
Based on your screenshot here, it looks normal, one backup per day:
I'm waiting for your diagnostics report before knowing what it could be.
If the backup file fails, it'll retry again.
0 -
@MikeT what I saw yesterday evening was a Dropbox lock file message in the icon tray for :
2018-03-24-21-27-08.zip
Less than 5 minutes later, I am getting another back up zip file being created and locked per Dropbox with a new filename:
2018-03-24-21-49-46.zip
The earlier file 2018-03-24-21-27-08.zip had been deleted five minutes later when the new file was created.
I had to leave 1PW7 today and work in 1PW4 because the lagginess was costing too much time. Trying to get taxes done :-(
Once I return to 1PW7, I will try to get screen shot evidence of this multi-back up zip per day phenomena.
0 -
@MikeT More testing this evening. If you recall in an above screen print, you can see opvault.backup having created back up file for today, 2018-03-26-12-03-43.zip
I gave up on 1PW7 in the morning, and did work out of 1PW4 while pointing to opvault. I made a number of updates to a new login entry, created and deleted a bunch of generated password, all in in 1PW4.
Finished my personal work, and returned to 1PW7 testing this evening. I opened 1PW7, and found an entry from earlier this morning that needed to be deleted. Pressed Ctrl + Del to delete a login record, entry was successfully deleted, and 1PW7 immediately updated a file called band_D.js and kicked off a new back up file. New back up file name 2018-03-26-18-44-12.zip. I think the last six digits in this filename is a time stamp?
My guess from the above series of events, 1PW7 is deleting any back up zip file created earlier in the day (?) then creating a new backup zip file for each new / change / delete event in the vault. While typing this email and not performing any action in 1PW7, Dropbox has shown no file changes.
I will further test this by creating a secure note now.
Now, the second file 2018-03-26-18-44-12.zip no longer appears in the opvault.backup directory and has been replaced by 2018-03-26-19-02-14.zip. The presumed time stamp matches up with my test activity.
I'm pretty confident with my assessment that any vault entry change is creating a new back up file.
If normal usage is a single entry change followed by a 5-10 minute pause in usage, this back up behavior is kind of invisible to the end user. If numerous changes are being made in quick succession, I am wondering if that might cause the app to lag and freezeas 1PW7 tries to amend what it is writing to a back up file?
0 -
Yes, that is correct. Every change you make triggers a backup and deletes the older backup on the day. You may see a few performance hiccups as it does this because you have so many attachments as 1Password compresses nearly 100MB of them per change.
We're looking into various optimizations to speed this up. One of which is to update the backups process to move the backups to its own dedicated folder within 1Password directory. We also plan to change how backups work in general, so that it is not recreating the same vault every time you make a change.
What you can do temporarily is copy the vault out of Dropbox folder, open that in 1Password and see if it works better with multiple changes in a row. I don't think it'll resolve all of the performance issues but the impact from Dropbox involvement should reduce it.
In your diagnostics report, some of the sync took over 130 seconds to read your entire vault for updates (it reads attachments too). So, it's not ideal right now and we're looking into optimizing the reading method to handle your type of vaults, read the items first, then attachments only if it is truly changed.
0 -
@MikeT I will do so. Yesterday I had to set up a second account with irs.gov (for a stupid reason that is entirely their issue, a horrible website experience) and 1PW7 was behaving so poorly that I returned to 1PW4 for safety. 1PW7 searches were returning the right number of entries, but click selections didn't seem to register correctly, and the rightmost panel wasn't refreshing to display the selected entry, so I couldn't be sure I had selected the right entry (old irs account or the new one) for editing. I use 1PW's password generation to create bogus answers to identity challenge secret questions, so it's imperative I get the data into 1PW and correct. Given that I couldn't be sure which entry I was looking at while editing, I had to go back to version 4 to make sure what I was doing. The work flow getting the data into the new 2nd irs.gov entry entailed many new generated password records, rapidly followed by multiple edits and saves to the #2 irs.gov entry. 1PW7 just could not keep up with that level of use. This is kind of a showstopper issue for me, so I do hope you guys figure it all out.
I'll look at temporarily moving the vault, but I prefer to keep it in Dropbox for Dropbox's offsite back up.
Out of curiosity, which section in the diagnostic report can I find which entries are choking the sync? If I can, I might take the time to tidy up those entries with file attachments. Making sure I've used low res images, compress files, etc. That may be a better long term strategy.
0 -
Out of curiosity, which section in the diagnostic report can I find which entries are choking the sync?
You won't find it as we do not log that kind of data. There's nothing in your database choking the sync, it's your vault itself. It is not actually typical to have a 100MB vault. The problem is that we read all 100MB to detect changes and then merge it into the local database and when you make changes, we update OPVault and then re-compress the entire 100MB into a backup next. Thus 1Password 7 is just not well optimized for your setup. 1Password 4 doesn't do this, it has no internal database and can just read/write straight to the specific band files without the attachments being involved.
For the moment, I would recommend not using 1Password 7 until we have an update that works better with your vault size. I will follow up here when we have an update ready.
I'm going to try to create a super-large vault for our engineering team to work with and we'll then figure out how to optimize it.
0 -
@miket For comparison, Mike, I have what may be a more 'normal' vault: 1,000 items, most of them passwords, hardly any attachments, 2 MB in size, and I see a lag also when I first open the app/mini. It's not making a backup, but something is going on that it takes too much time to fill the first login fields.
I'm already on the page, it knows what to do, it's just the fact that it's the first unlock/processing of the day. And it's slower than 1P 4, I'm pretty sure. Is this a known issue that will be optimized?
0 -
Hi @leesweet,
That's a separate issue, that's the network activity (database init, proxy check, sync, etc) blocking the UI from doing its job. It will happen every time you unlock the app and every time there's a sync in the background.
It will be optimized yes but no timeframe. We most likely will not work on this until we finish most of the features first for 1Password 7.0 release.
0 -
Gotcha, will deal while you get the rest of the work done!
0 -
@MikeT Okay, gotcha. I will stand by. Thanks so much! I probably will still do light playing around in 1PW7, and keep any heavy lifting in 1PW4
0 -
@leesweet, @Superfandominatrix: Likewise, thanks to both of you for your patience and pushing us to improve the beta. :)
0 -
I thought i was doing something wrong but glad to see it’s in the works. Thanks for all the hard work!
0 -
@Greg @brenty @MikeT Retested this matter on 1PW7 v7.0.539, I don't have a baseline in terms of timing for the prior version, but my gut says this is slightly better. I tested by starring and then trying to unstar the same entry.
I am coming in at a 7 second count between starring, and then a successful click unstar. Better, but not great.
0 -
That's great to hear, we haven't started to address this yet, the move to our own backups directory helps a bit but it is not the cure for sure. We still have a lot of optimizations planned prior to the stable release of 1Password 7.0.
0
