Update on migrating from mSecure 6 [Solved]
I'm working to migrate and consolidate Lastpass and mSecure 6 into 1password. It was a little bumpy getting Lastpass to work but got it with the csv file.
mSecure is a harder nut to crack (mSecure Version 6.0.8 (896)). The exported csv is a mess since I used a couple of entry types the fields don't line up. It seems to be exported with the name of the field and number coded like "Password|8|" and "Username|7|". Also every entry starts with the name and entry number like "Example entry|1531018355," and so forth.
Tried using MRC Converter Suite (mSecure 5) but got an uncaught error on validating date around line 200. Might want to work this further if MRC is interested.
At this point, I have the Lasspass info to start from but my mSecure data is more recent. So this becomes a manual process.
I'm wondering when importing the same entry from Lastpass and mSecure. What does 1password do with similar entries? Does it attempt to resolve conflicts or just overwrite? Does it offer any control to the user to resolve?
Thanks for any help on making this easier.
... Mark
1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Big Sur 11.7.3
Browser:_ Chrome
Comments
-
Appreciate you taking a look. Here is the command and the error messages. Thanks.
... MarkMSL-MacBook-Pro-Late-2013:mrc-converter-suite Mark$ perl convert.pl msecure5 -v mSec6.csv
Can't call method "year" on an undefined value at
/Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 477 (#1)
(F) You used the syntax of a method call, but the slot filled by the
object reference or package name contains an undefined value. Something
like this will reproduce the error:$BADREF = undef; process $BADREF 1,2,3; $BADREF->process(1,2,3);Uncaught exception from user code:
Can't call method "year" on an undefined value at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 477.
Converters::Msecure5::date2monthYear("Expiration Date|11|11/2024") called at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 151
Converters::Msecure5::ANON("Expiration Date|11|11/2024", "expiry") called at /Users/Mark/Downloads/mrc-converter-suite/./MCS/Core.pm line 151
MCS::Core::normalize_records(MCS::Core=HASH(0x7fb241232440), MCS::Generators::PUX=HASH(0x7fb2404985d8), "creditcard", ARRAY(0x7fb24029e780), HASH(0x7fb24029e768)) called at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 379
Converters::Msecure5::import(Converters::Msecure5=HASH(0x7fb240826da0), "mSec6.csv", undef) called at convert.pl line 200
MSL-MacBook-Pro-Late-2013:mrc-converter-suite Mark$0 -
Ok, this is an easy one to spot.
The converter is trying to convert the credit card's month / year expiration value, but it is seeing the entire string "Expiration Date|11|11/2024" as being that month / year. Since the mSecure CSV export dos not contain any labels, your data must have that label / value as the date data. And clearly that's isn't a "date", let alone a month / year value.
Would this data have been imported by mSecure from some other password manager, or did you add that? I thought I recall that mSecure5 validated date input, but perhaps it doesn't on importing from another password manager.
Either way, I recently updated some of the date conversion routines to better handle such egregious data (normally I like to know when a password manager can create bogus data so that I get reports like yours).
I just updated the development (not release) version of the suite. Can you retrieve that and give it a try? It's in the folder mrc-converter-suite - development version at top level of the download site. Follow the mrc-converter-suite home link here:
https://1password.community/discussion/101693/moving-to-1password-from-another-password-manager
Close your terminal window, toss your existing mrc-converter-suite into the trash, replace it with the unzipped development version I just posted, launch your terminal again, change directories and retry the conversion.
Here is my run with your bad month / year value:
$ perl convert.pl msecure5 /tmp/msecure5_mac.csv -v Import summary: 11 items discovered Export summary: 1 Bank Account item 1 Credit Card item 1 Email Account item 1 Reward Program item 2 Login items 2 Membership items 3 Secure Note items 11 total items You may now import the file below into 1Password 8: /Users/cappella/Desktop/MCS_converted.1pux0 -
Excellent, I'd be happy to give it a try. The csv file is as exported by mSecure 6. There are no controls provided to alter the export. I haven't tried importing to any other. I'm a bit perturbed that mSecure 6 exports in this fashion, but they may have their reasons. Doesn't make it easy, however.
0 -
Hmm. That went a lot better ...
MSL-MacBook-Pro-Late-2013:mrc-converter-suite2 Mark$ perl convert.pl msecure5 mSec6.csv -v
Use of uninitialized value $val in concatenation (.) or string at
/Users/Mark/Downloads/mrc-converter-suite2/./Converters/Msecure5.pm line 352 (#1)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.To help you figure out what was undefined, perl will try to tell you the name of the variable (if any) that was undefined. In some cases it cannot do this, so it also tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo, and the warning will refer to the concatenation (.) operator, even though there is no . in your program.
Import summary:
905 items discovered
Export summary:
3 Bank Account items
3 Membership items
3 Reward Program items
15 Credit Card items
311 Secure Note items
570 Login items
905 total items
You may now import the file below into 1Password 8:
/Users/Mark/Desktop/MCS_converted.1pux
MSL-MacBook-Pro-Late-2013:mrc-converter-suite2 Mark$So I am successfully able to create the 1pux file. Nice, this is progress!
However, I go to import into a new Test vault and can't seem to import the new file (see snap below). When I select import from the desktop application it only allows me to import directly from Lastpass. This never worked for unknown reasons -- I got it to work from the web interface.
It looks like I should be able to feed it a file but the pop-up never allows me to select the "From File" tab. Hmm, this looks broken to me. Any suggestions?
!
0 -
Yes, that is progress.
The error might be innocuous. It's from a debug message. If the import works, then there's no need to worry about it.
The import dialog where you pick LastPass or From File is goofy. Click on LastPass and it will switch. Then you can drag your 1PUX onto the drag area.
0 -
Right, click Lastpass to import from file -- a little counter-intuitive but you are right -- this allows me to import into a new vault called "Imported M_C_S". Cool, that works.
The real problem is with the conversion of mSecure 6 data. Since the mSecure 6 export format seems to label|field#||value pairing for everything. The imported value for some usernames becomes "Username|7|testuser" for example. Passwords imported look like "Password|8|BogusPassword" which is not what was intended. The label such as "Password|8|" needs to be removed and just the remaining part of the string kept for the password.
Similarly, entry names are formatted as title|ID for example "Cool Website|0123456789" which gets imported as a tag value. The title doesn't get used as the name of the 1password entry. Major bummer.
I can't think of any reasonable way to fix this. I'd sure love it if your conversion tool handled this right! I believe this impacts everyone coming from mSecure 6 to 1password. Thanks.
... Mark
0 -
Thanks for your help. I'm happy to assist with sample data and all. Tomorrow is good.
0 -
FYI: The mSecure 6 export format requires a completely new converter module (msecure6) to accommodate the large changes in the export format. I'm working on this now.
Edit: Question, have you customized / changed any of your items stock field labels?
0 -
Progress...
0 -
Thank you for working this! The main entry types/labels that I use are: Login, Computer Login, Credit Card, Serial Number, Secure Note, Account, Prescription, Combination, Frequent Flyer, Bank Accts, Frequent Flyer Hotel, Email Accts.
Somehow along the way there were duplicates created like: Web Logins Duplicate, Credit Card Duplicate, Combinations Duplicate. I believe they are similarly but the fields move around a bit. Other than this, I didn't modify the structure at all just used it.
This doesn't have to be a 100% solution but something get close would be most helpful. Thanks again.
0 -
In order to get you a converter as quickly as possible, I'll focus on what you need for your data, and will flush out the converter later.
I'm working first on the stock Templates (and these will map to 1Password "Categories"). I've completed Login, Credit Card, Secure Note, Bank Account. I'll now add the other stock templates you list (Prescription, Combination, Frequent Flyer, Email Account).
For your custom templates, I'll need to know how you want them mapped into 1Password's categories (without this, they will end up as Secure Notes and many will anyway). Your custom templates are: Computer Login, Serial Number, Account, Frequent Flyer Hotel.
The converter currently is matching the field labels within each item to map these into 1Password's field's for a category. So if you've changed those labels, we'll have to add the appropriate pattern into the converter's field label pattern match definition. You'll know this when you examine the results, and we can adjust accordingly.
0 -
In the Email Account template, what is the difference between these two fields?
0 -
Is the "Name" field in Prescription a person's name, or a medication name?
0 -
Sound like a great approach. To answer your question about the custom templates. I'd suggest this mapping of custom template to 1password category
Computer Login --> Login
Serial Number --> Software License
Account -- Drop this as too vague (fixed my one entry)
Frequent Flyer Hotel --> Rewards ProgramQuestions:
Any 1password category for Insurance Info? (included snap of mSec6 types)
We can iterate to polish this up for general usage. Thanks!
0 -
OK on the mappings. I need to know the exact field names of those templates (case sensitive).
Regarding Insurance info, I have most converters map these into Membership, and have the converter add custom fields for things like Group #, Policy #, etc. In fact, I'm already handling this:
So, as soon as you provide me with the field labels for each of those custom templates, we can start testing your new conversion.
Also, see my other questions above too.
0 -
Sorry, missed I some of your email while I was look at this ...
When I enter a new Email account, it presents the SMTP host field. I don't see the Outgoing Mail Server field. Perhaps it was deprecated. It is equivalent.
The Prescription Name field is the medication name.
Computer Login:
Userid
Password
Custom 4
Custom 5
Custom 6Serial Numbers:
Serial #
Purchase Date
Reseller
Field 5
Field 6Frequent Flyer Hotel
Number
Password
URL
Custom 5
Custom 60 -
Sorry for the delay. I wanted to test on both Windows and macOS, since the CSV export differs based on platform.
The Email account mapping may be wonky. The 1Password Email Account template was built when POP email and dial-up ISP access was common. If you don't like the mappings, let me know.
I've updated the dev. release so you can test. The converter's name is msecure6. So use it as in:
perl convert.pl msecure6 -v ..\msecure_export.csv
You can add the
--debug(or-d) option if you want to review the mappings without importing into 1Password.Some notes on your custom templates:
I am not mapping the Custom # fields; they will go into the Notes area. If you want them mapped, let me know.
The Serial # and Reseller fields for your Serial Numbers template does not map naturally to any fields in the Software License category, so they will become custom fields. I don't know what data type your Purchase Date is, and I've presumed it is a Date Field type (which has a date picker and ID value 10 in the CSV export (the middle value in the pipe-separated value, as in
label|id|value). This is important because the date value is in a specific format that gets converted to 1Password's internal representation for date values for date fields. If your Purchase Date is just text, I'll redefine that definition to not attempt to convert this value.I have not created custom templates identical to yours, so haven't tested with any data - you'll have give me feedback on the mapping.
I'll await your feedback.
Edit: I updated the dev version to resolve an issue with placing empty string tags in each item.
Edit: I updated the dev version so that the README is now updated with this new msecure6 converter.
0 -
Excellent. I'll give it a shot on Thursday. Thanks!
0 -
Conversion went pretty well ...
MSLMBPLate2013:mrc-converter-suite-2B Mark$ perl convert.pl msecure6 -v ./mSecure6ImportA.csv
Import summary:
817 items discovered
1 items were added by splitting 1 original item
Export summary:
1 Medical Record item
2 Software License items
3 Bank Account items
3 Membership items
7 Reward Program items
13 Credit Card items
206 Secure Note items
583 Login items
818 total items
You may now import the file below into 1Password 8:
/Users/Mark/Desktop/MCS_converted.1puxLoaded into 1password and looked pretty reasonable. Didn't look bad at all. Looks like a lot of things turned into Secure Notes, however. Could it be that all the Duplicate types turned into Secure Notes? I'll get back to you on Friday with a better analysis of what happened.
Once something gets categorized, can I edit it to change the categorization? I can do this in mSecure. Thanks!
0 -
Good progress!
Categories cannot be changed, so its worth getting this right w/the converter. Examine a few big hit categories that you feel aren't correct, and we can adjust to hit the majority.
Dup's don't cause incorrect categorization - the category is defined in the CSV, and the converter maps that to a 1Password category where it can. Some mSecure templates don't map naturally, so they may end up as Secure Notes.
0 -
Another pass on this and it looks pretty good. Here is the output ...
MSL-MacBook-Pro-Late-2013:mrc-converter-suite-2C Mark$ perl convert.pl msecure6 -v ./mSecure6ImportD.csv
Import summary:
817 items discovered
1 item was added by splitting 1 original item
Export summary:
1 Email Account item
1 Medical Record item
1 Passport item
2 Software License items
4 Membership items
5 Bank Account items
13 Reward Program items
15 Credit Card items
49 Secure Note items
727 Login items
818 total items
You may now import the file below into 1Password 8:
/Users/Mark/Desktop/MCS_converted.1puxI may do some more converting but I think I'm close enough. Now this becomes a manual process of cleaning everything up. I'm now able to see all entries from Lastpass and mSecure6 together in 1password.
MRC -- Thanks so much for jumping on this and providing the conversion tool. You made this process doable and saved me hours! I really appreciate it. Other mSecure 6 guys to come will thank you too.
All the best.
0
