Question regarding backup setting, and other things

ScarySulley

In 1Password 4, your data (for all vaults) is always stored in a default location. For the Mac App Store, this location is: ~/Library/Containers/2BUA8C4S2C.com.agilebits.onepassword-osx-helper/Data/Library/Data/OnePassword.sqlite, and the location for the webstore version is : ~/Library/Application Support/1Password\ 4/Data/OnePassword.sqlite

Please note that this data cannot be moved. When you choose to sync your data, 1Password will actually make a copy of this data and store it in the appropriate sync location. Then both the local copy and the sync copy are updated when changes are made. This is an improvement over 1Password 3 where your data was only stored in one place - there were far too many cases where a 1Password.agilekeychain file would accidentally get deleted from Dropbox, forcing a user to restore from a backup.

Interesting about having two copies that are synced if you use DropBox. There were some odd issues where sometimes the file on DropBox would get deleted or something when I was testing with 1Password 3. Good to know there's a local 'back up' file.

So the encrypted OnePassword.sqlite is equivalent to the encrypted agilekeychain found in DropBox and 1Password 3? Lastly, what is the OnePassword.sqlite-wal file for?

Thanks again Megan! :)

Megan

Hi @ScarySulley,

So the encrypted OnePassword.sqlite is equivalent to the encrypted agilekeychain found in DropBox and 1Password 3?

Yes and no. It contains the same data: your 1Password database. However, in 1Password 4 we're moving towards a new keychain format, so the encryption on the OnePassword.sqlite is slightly different. You can read all about the technical details here: The continuing evolution of security in 1Password 4. The .agilekeychain format is still being used in Dropbox (and Folder Sync) to ensure that we have backwards compatibility for the platforms (such as Windows and Android) that do not yet have 1Password 4 and thus cannot read the new format. Once 1Password 4 is released across all platforms, we'll begin the process of fully switching to the new cloud keychain format.

Lastly, what is the OnePassword.sqlite-wal file for?

This file helps to track changes for sync. :)

ScarySulley

Hi Megan,

Getting to know 1Password 4 a bit more and have some questions:

---

1) Updating data not reflecting in Finder

When I update data, the .sqlite file (or agilekeychain in Dropbox or Folder Sync) doesn't update right away. What I mean by that is the date modified doesn't change -- since I started this post, the modified date has changed however, but that was an hour after the last modified date. Does the modified date/time not change even if the data has? I would think it would be update "live." But I'm guess I'm incorrect.

---

2) Multiple vaults

Regarding multiplie vaults...there's the main vault in Application Support/1Password 4/Data/Onepassword.sqlite...Then if you sync via Dropbox, Folder Sync, iCloud, etc., you have essentially have two data files as you told me earlier.

Now let's say you create another vault. I'm assuming the main vault contains all vaults. When you sync another with Dropbox, etc., that vault's data is 'extracted' from the main vault, and put into the sync Dropbox folder, under it's own name? Right now I synced both vaults (primary and secondary) to DB, and both each have their own agilekeychain file. So I'm guessing I'm correct...(I was testing as I was typing, so I might as well leave the text here ;) )

---

3) Changing details in non-primary vaults

I was doing some research on how to change the details on editing a non-primary vaults details. I read this tutorial. I made a test vault and synced it to a folder on my desktop. Then copied that file in that folder to the desktop. I then double clicked that file to make a new vault (and hence edit it's details). However, I unable to add a new password. It only allows me to enter the original password of the test vault I made.

---

4) Unknown file type

Under the main directory (Application Support/1Password/...) I noticed besides the two files, sqlite and sqlite-wal, I mentioned earlier, I see a new file called "OnePassword.sqlite-shm." What does this file do? :)

---

Thanks again (really I mean that) for your assistance as I get to learn 1Password 4.

Megan

Hi @ScarySulley‌,

Always happy to help with the questions! I'll admit, I'm learning a few new things with some of these technical questions. :)

1) Updating data not reflecting in Finder

The reason for this delay in the update to the .sqlite file itself is that the database change is written first to the Write-ahead Log (the OnePassword.sqlite-wal file we talked about earlier) then a checkpoint occurs and the change is written to the .sqlite file.

2) Multiple vaults

You're right, your sqlite file in the Application Support folder will contain all of the data from all of your vaults. When you sync a secondary vault, 1Password is 'copying' (not so much extracting or removing) the secondary vault's data to its own separate sync location. (As you determined.)

3) Changing details in non-primary vaults

The tutorial you mention is primarily to change the colour of the icon and name of the vault, the incidental details. If you would like to change the Master Password on the vault, you will need to export all the data in that vault and create an entirely new vault with the Master Password and details you want, then import the data. Please note that exported data in .1pif format is not encrypted, so you want to be sure to delete it properly after it has been imported.

4) Unknown file type

The OnePassword.sqlite-shm is another helper file of the main OnePassword.sqlite file. If you want to read all of the technical details about this and the -wal file, you can find definitions and details here: https://www.sqlite.org/

ScarySulley

Thanks for the info again!

In a test I'm doing, I deleted the 1Password 4 folder to see what would happen. For some reason 1Password is still asking for my password when all data files and folders under Application Support have been deleted, including the Preferences files. The 'logins' and 'Trash' on left pane have a bunch of numbers over them and it won't let me add data. I know this is because I 'messed it up,' which is fine for now. But where is it getting the password from when the data has been deleted? Is there some data it keeps elsewhere? Dropbox data has been deleted.

sjk

Hi @ScarySulley,

1Password can only be prompting for a password if it's finding data somewhere to unlock. :)

Seeing a Diagnostics Report from your Mac would be the easiest, quickest way to determine where that data currently is. And if you go through instructions in the Starting over guide for 1Password 4 to remove its local database then the following setup might detect 1Password data and show its location.

ScarySulley

Hi sjk,

I've sent the report to support. I'm really curious as to where 1Password is getting that data from. :)

Thanks again!

Megan

Hi @ScarySulley,

Someone from the team will get back to you directly via email about this. :)

.

ScarySulley

Hi Megan,

I think I have this figured out but wanted to double check my understanding regarding import/export and backup/restore.

Import/Export

The file(s) produced from Export are not encrypted (as the window warns). The data in this file can be used to merge into another existing vault. It can not be used to restore a vault -- File > Restore or using "I've used 1Password before" when starting 1Password fresh.

Backup/Restore

The file(s) produced from Backup are encrypted. These files can not be used to merge data with another existing vault. Their purpose it to restore any data that may have been deleted in a vault that it belongs to. They can only be restored when starting fresh or when restoring from it's 'parent' vault.

Possible bug?

It also seems as though when the backup files are moved from their default location 1Password gives me an error message if I try to restore from a backup file I put on my desktop. And if I get click 'Find Backup' (under File > Restore) I also get the failed error message after I click 'Add backup' (from default location). Maybe this is a bug? It seems as though the Restore function is finicky as to where you put and how you handle the backup files. Now I get a 'backup failed' notification after trying that 'Find Backup' option...hopefully that makes sense. I was testing and typing as I went...

haha I like how this started out just confirming something, and now I may have discovered bug :( oh well I'll part of fine tuning things! :)

ScarySulley

So regarding the 'bug' and to make the above a bit clearer after some more testing...It seems as though I only get the 'Failed error' if I use the Find Backup button to find a backup file. If I simply select a backup file from the 1Password window, it works fine. Again, I'm not sure if this is a bug, or if I'm doing something wrong. :)

sjk

Hi @ScarySulley,

Thanks for the detailed writeup about these items.

Import/Export

That's correct. Although it's something we may consider, allowing 1PIF data to be imported during setup and/or vault creation could encourage more usage of that unencrypted format which can leave 1Password data vulnerable.

Backup/Restore

Also essentially correct. Backups contain the complete 1Password 4 database, with all its vaults. Selectively restoring specific vaults from backups may be possible in the future.

Possible bug?

Correct enough! :)

Using Find Backup to add a backup always displays the Failed adding backup error even after it's successfully added. I've filed a bug report for that.

It also seems as though when the backup files are moved from their default location 1Password gives me an error message if I try to restore from a backup file I put on my desktop.

How exactly are you trying to restore a backup file directly from Desktop? It would first have to be added to the restore list using Find Backup (displaying the aforementioned error after success), then selected/restored from there. Or am I overlooking something? :)

Thanks again!

ScarySulley

Hi sjk,

Interesting on the bug. I'm normally not this concerned with software, but considering the nature of it and seeing as how you all interact with customers well, I like to bring any possible issues to attention of the developer.

How exactly are you trying to restore a backup file directly from Desktop? It would first have to be added to the restore list using Find Backup (displaying the aforementioned error after success), then selected/restored from there. Or am I overlooking something?

I did several test:

1) Restoring a backup file from the actual window without clicking on Find Backup. I just clicked an item (that was in the default Backup folder) on the list then clicked Restore. That went fine.

2) Same as above in regards to the default location expect this time I used Find Backup that opened Finder and navigated to the Backup folder. This resulted in the Failed Error. Although I did not relaunch 1Password to see if it restored regardless.

3) Same process as #2 (clicked Find Backup and went through the Finder) but this time the file was in a folder on my desktop. I copied the backup file from the default backup folder to the folder on the desktop (I moved it just to see what would happen). So I would click Backup from the File menu to get the backup file. Again, I did not relaunch 1Password to see if it restored regardless.

Hopefully that clears things up for you. :)

sjk

Hi @ScarySulley,

Thanks for that clarification.

The way Find Backup works is by first adding one you've selected (e.g. from Desktop or some other location) to the 1Password 4 Backups folder (if possible), which then appears in the Backups list of the Backup tab of Preferences to select for restoring.

The bug with Failed adding backup always being displayed, whether the backup has actually been added or not, is fixed in the latest beta (currently 4.3.BETA-15). Using Find Backup to add any backup that's already in the Backups folder (typically with automatic backups) is always going to fail (and correctly display the message) because it already exists there.

All clear for you now, too? :)

ScarySulley

Hi @sjk,

Ok let me see if I understand this regarding Find Backup:

1) You click Find Backup and locate a backup file (eg: from the desktop or elsewhere)

2) That will then copy/move the backup file to the Backups folder in which is can then be chosen from the Backup list. In other words, 1Password can't restore a backup from any other folder besides the main Backups folder.

You mentioned "if possible" when adding a backup file from the desktop/other location back to the Backup folder. What would make that not possible? :)

sjk

Hi @ScarySulley,

Yup, you've got it!

It's not possible for Find Backup to add (copy, not move) a backup file to the Backups folder if it already exists there, like you'd attempted earlier:

… this time I used Find Backup that opened Finder and navigated to the Backup folder. This resulted in the Failed Error.

That's a case when Failed adding backup really means what it says. :)

ScarySulley

Thanks for the info...gotta another question: :)

I found these instructions how to change a logins password. When I do the steps, the password history isn't quite the same as 1Password 3 (I couldn't find anything for 1Password 4 after a quick search, unless I missed it). When I'm on a particular site, it does show a history that shows a list (URLs that are shortened), but when I click on them, nothing happens. How exactly does one properly change a password via the browser plugin? I know I could copy the new password before I click FILL, but I'm curious about the History thing.

Thanks!

EDIT: Ok I just went to the main app and saw new items listed under "Passwords." I'm assuming that I just copy that password to the existing login entry for that login?

sjk

Hi @ScarySulley,

Here's the 1Password 4 guide for changing passwords:

Updating your site's password

If you have questions it doesn't answer please let us know. :)

For more about History list truncation in 1Password mini's Password Generator see:

Password Generator – Can't tell which password is newest

ScarySulley

Hi @sjk,

Thanks for the info. Glad to see updating logins is so seamless.

Question regarding exporting. 1Password 4 for Mac has three options (none of which are encrypted):

• 1PiF file

• and the two text files that separate the data either with tabs or quotation marks.

Now I know that the 1PiF file can import data into 1Password and thus merging the data. But what are the other two for besides having them as an offline backup? Can you import them as well? And is there any difference between the tabbed separated file verses the quotation marks?

Lastly, and I don't remember if I asked this before, but are the Backup files in the backup folder (as opposed to the Export options) encrypted?

Thanks!

Jasper

Hi @ScarySulley,

But what are the other two for besides having them as an offline backup?

If you want to get your data out of 1Password, you can export to standard file formats such as CSV and TXT. We don't want to lock people into our product. Your data is your data, which is why standard export options are available. If you aren't planning on moving your data out of 1Password, these export options likely aren't for you.

And is there any difference between the tabbed separated filed verses the quotation marks?

The tab delimited option means that each piece of data is separated with a tab, in a TXT file. The comma delimited option means that each piece of data is separated with a comma, in a CSV file.

Can you import them as well?

1Password can import from a CSV file, but not a TXT file. If you're exporting for the purpose of importing back into 1Password, you should use the 1PIF option.

Lastly, and I don't remember if I asked this before, but are the Backup files in the backup folder (as opposed to the Export options) encrypted?

The backup files themselves aren't actually encrypted, but their contents are. Each backup file (.1p4_zip) contains a copy of your OnePassword.sqlite file, which is encrypted.

ScarySulley

Hi @JasperP

Thanks for the information.

1Password can import from a CSV file, but not a TXT file. If you're exporting for the purpose of importing back into 1Password, you should use the 1PIF option.

Since 1Password can import from both a CSV and 1PIF file, what are the differences (if any) between the two?

sjk

Hi @ScarySulley,

The 1Password Interchange Format (1PIF) is a file format developed by AgileBits specifically for import/exporting 1Password data, primarily with 1Password itself.

Comma Separate Values (CSV), aka Comma Delimited Text in 1Password, is a general purpose file format. Wikipedia's Comma-separated values page is a good source of information about it. And 1Password also supports exporting (but not importing) the related Tab-separated values (TSV) format, aka Tab Delimited Text:

Significant differences and intended/possible purposes. :)

ScarySulley

Hi @sjk,

Besides 1PIF being a format developed by AgileBits primarily for 1Password and CSV being the general purpose format (sorta like a camera native RAW format compared to a DNG file), the end result is the same, right? All the same data will be imported...? :)

P.S. You all reply really fast. I usually come back 24 hours later and there's already apply from the previous day! :D

ScarySulley

@sjk,

Just bumping this back up incase it was overlooked.

Still curious as to the differences between 1PIF and the CSV file since they can both import the data. :)

Also, is there a difference between 1Password mini and the browser plugin?

Megan

Hi @ScarySulley‌

I do apologize for the delay here! We've been a bit busy since the discovery of Heartbleed, but we certainly didn't mean to overlook your question.

Still curious as to the differences between 1PIF and the CSV file since they can both import the data

The primary difference between the .1pif format and the .csv format is that we designed .1pif to work specifically with our data format. Using a .1pif file to export and import your data will ensure that all your data in each category will be copied correctly. As mentioned in the import guide, CSV import only supports importing Logins, Credit Cards, Software Licenses, and Secure Notes.

Also, is there a difference between 1Password mini and the browser plugin?

1Password Mini does all the processing and heavy-lifting (for both the main app and the extension). It truly is the brains behind the operation. The browser extension is only responsible for the actual filling of data into the website. This ensures that there is no lost data between the extension and the main app because they are both pointing to the same database via the 1Password Mini. :)

ScarySulley

Hi @Megan,

1Password Mini does all the processing and heavy-lifting (for both the main app and the extension). It truly is the brains behind the operation. The browser extension is only responsible for the actual filling of data into the website. This ensures that there is no lost data between the extension and the main app because they are both pointing to the same database via the 1Password Mini.

So they are different, but not in a way the end user would need to know. With that said, couldn't we just use 1Password mini and not have the browser extension(s) installed since they seem to both to the same thing? When I click on either one, they both have the same UI. Again, I know you said they are a bit different, but to the end user they seem the same. Although I'm not sure if 1Password 3 for Windows has Mini, as I haven't installed it yet so the extension might be the only quick way to access the database as opposed to the full app.

Also, under Chrome, I have syncing of extensions disabled, as the Mac version and PC version are different, so I get a conflict type of message when the Mac installs the 1Password 3 extension the PC also has installed (and vice versa). Both end up being added to each platform...if that makes sense. Is there any way to get around this other than disabling extension syncing since Mac has 4 and PC has 3...?

Thanks! :)

sjk

Hello again, @ScarySulley. It's me again this time. :)

The 1Password 4 browser extension provides "connection glue" between web pages and 1Password mini. 1Password 4 autofill/autosave features require the extension.

1Password mini and what you see when you use the browser extension is the exact same thing. 1Password mini's UI is the front end to the browser extensions. The browser extension itself is required to integrate with the browser, to look at the site's source code and allow for the auto-filling to work, in addition to submitting the form, and detecting Login forms to save data for new Login items.

At the moment, disabling Chrome extension syncing is the workaround for the conflict issue you've described. The upcoming upgrade for 1Password for Windows from version 1 to version 4 will make extension syncing possible again. We're leapfrogging past unreleased versions 2 and 3. :)