Bug in MRC Conversion Tool - Firefox and Wrong Timestamp [Solved]

AndreasW2

Hi MRC

since I am new to all this I did not know what the best place was to post it, hope it is okay here.

First of all thank you for your conversion tool, it literally saved me hours today.

So I thought it is only fair that I can give back what I can: a bug report :blush:

I am running 85.0.2 (64-Bit) on Windows and used the latest version of your scripts to export/convert the Firefox Passwords to an 1PF file. However, the import on the most current 1Password 7 Desktop Version failed, because in the converted file, the timestamps where somehow wrongly formatted - they were a decimal, not a float, for example like this:

"updatedAt":1514482148.092

Obviously, the ".092" in the end failed the import.

When I ran the script with --no-timestamps, the import worked.

I hope this is reproducable and thanks again
Andreas

---

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

MrC

Hello @AndreasW2

You're welcome, and thanks for the report.

Can you tell me how you exported your Firefox passwords?

Also, which version of Firefox are you running?

I just tried it's native method to export as CSV but don't see floating point values for the time stamps, but rather integer stings.

AndreasW2

Hi @MrC

I am running Firefox 85.0.2 (64-Bit) on Windows and I exported it like in the readme, via "perl convert.pl -v firefox" - then I said yes to export my online profile, entered the master password, got confirmation of xxx imports and exports and that is that :)

Thanks
Andreas

MrC

@AndreasW2 ,

Ok. The reason I asked is that Firefox has changed its encryption scheme, and the native firefox converter no longer works. So I'm wondering why it worked for you. I'm guessing that the new scheme only takes affect for new profiles, or those who just start using saved passwords in Firefox, or who have not allowed Firefox to refresh (a newer feature in Firefox which updates a bunch of databases if Firefox has not been used in a while).

So let's get onto the bug. When I wrote the script, timestamps were saved in milliseconds, and the converter divides by 1000 to resolve that. I wonder if that has changed. In the Converters\Firefox.pm file, at line 211, you'll see the code:

$_->{$datekey} /= 1000;

Can you try commenting that out, as:

# $_->{$datekey} /= 1000;

and re-running the converter, and check the dates?

MrC

@AndreasW2

Just checking in with you to see if you need any more assistance.

AndreasW2

@MrC sorry for the late reply, I was busy doing work stuff :)

Yes, commenting out that line works like a charm - the dates are all right now.

Thanks
Andreas

MrC

Cheers, thanks for the update @AndreasW2 .

Enjoy 1Password!

Dayton_ag

Thanks for all your help, @MrC! :smile: :+1:

Tertius3

@Mrc did you change the Firefox importer yet? I just converted from Firefox and encountered the issue with the float timestamps in the generated file. I used the converter from mrc-converter-suite-2021-03-03-1143.zip.

The timestamps are still generated like "updatedAt":1311525370.593, ....

This is the correct time stamp value in seconds, but the fraction must not be. I wasn't able to find where/how the converter writes the *.1pif file, but it seems in Firefox.pm you're creating a float in line 211 and save this float value to %cmeta in line 216. If the module that actually writes the *.1pif file isn't truncating this value to int, it will write the float value in error. So please either truncate the value in line 211 of Firefox.pm or truncate the value wherever this value is written to the *.1pif file.

I fixed the output with sed -re s/At(.:[0-9]+).[0-9]+/At\1/g output.1pif > fixed_output.1pif but I don't think that's the intended way ;)

MrC

@Tertius3

No, I have not yet changed the firefox converter. My exports are still showing the timestamps in milliseconds, so I have not been able to reproduce the case you and the other user mention.

Which Firefox version and platform are you exporting?

Tertius3

It's the current version 87.0 64-bit under Windows 10. I exported from 2 Windows machines. Their password databases were created years if not decades ago. Both exports contained the erroneous timestamp values in the end.
As far as I see, my Firefox uses a password database file called logins.json that contains entries like this:
{"nextId":283,"logins":[{"id":1,"hostname":" [...] ,"encType":1,"timeCreated":1300900911900,"timeLastUsed":1300900911900,"timePasswordChanged":1300900911900,"timesUsed":1}

This is converted by your suite to:
{"updatedAt":1300900911.9,"openContents":{},"createdAt":1300900911.9,"typeName":"webforms.WebForm", [...]
5642bee8-a5ff-11dc-8314-0800200c9a66

You see Firefox timestamps are in milliseconds, but not 000 but 900 at the end, so this shows up as .9 in the export. I recommend you just truncate the result value of the division to integer after line 211 of Firefox.pm.

The correct 1pif entry must look like this:
{"updatedAt":1300900911,"openContents":{},"createdAt":1300900911,"typeName":"webforms.WebForm", [...]
5642bee8-a5ff-11dc-8314-0800200c9a66

MrC

@Tertius3 ,

What I'm trying to understand is a) why some of us can no longer use the firefox converter (due to the encryption changes) and why some can, and b) understand why these timestamps once divided by 1000 (in the Firefox.pm module) have always worked for others in the past, but now do not. I don't believe we've all been lucky with the milliseconds having 000 as the last three digits in the timestamp, across all of our entries.

So either 1Password has changed, or something that I don't understand has changed. That's what I'm concerned about most - not the rounding or truncation.

I don't add code if I don't understand how to, or cannot, replicate the situation to test it. And I have not been able to run the firefox converter yet (which requires the old encryption scheme) to understand how to obtain the result you (and the other user above) have been able to arrive at this issue. Since I've been so busy, I haven't spent time trying to install an old Firefox version in a VM to reproduce the case. My working theory has been that Firefox will use the old scheme until a user allows Firefox to "refresh* its data when prompted. Did you allow this,or deny it?

Tertius3

Unfortunately, I don't remember Firefox asking to "refresh" my data. It might have happened or not - I don't remember.

About the question what has been changed, perhaps on the 1Password side the import function changed in a way that it accepted float values in the past (and truncated the values), and now it doesn't do that any more and accepts only integer values. That's the only thing that comes into my mind what might have been changed, because the old Firefox database format wasn't probably changed for a long time. The other thing is some Perl module writing that value was updated to include and write fractions, but that's probably not the case, since such functions tend to be extremely stable to avoid breaking apps in a way we experience here.

Blake

Thanks a bunch for your help here @MrC! 💙