Update on migrating from mSecure 6 [Solved]

Options
netefx7
netefx7
Community Member
edited February 2023 in Lounge

I'm working to migrate and consolidate Lastpass and mSecure 6 into 1password. It was a little bumpy getting Lastpass to work but got it with the csv file.

mSecure is a harder nut to crack (mSecure Version 6.0.8 (896)). The exported csv is a mess since I used a couple of entry types the fields don't line up. It seems to be exported with the name of the field and number coded like "Password|8|" and "Username|7|". Also every entry starts with the name and entry number like "Example entry|1531018355," and so forth.

Tried using MRC Converter Suite (mSecure 5) but got an uncaught error on validating date around line 200. Might want to work this further if MRC is interested.

At this point, I have the Lasspass info to start from but my mSecure data is more recent. So this becomes a manual process.

I'm wondering when importing the same entry from Lastpass and mSecure. What does 1password do with similar entries? Does it attempt to resolve conflicts or just overwrite? Does it offer any control to the user to resolve?

Thanks for any help on making this easier.

... Mark


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Big Sur 11.7.3
Browser:_ Chrome

Comments

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    Tried using MRC Converter Suite (mSecure 5) but got an uncaught error on validating date around line 200. Might want to work this further if MRC is interested.

    Can you provide more of the error message so I can take a look?

  • netefx7
    netefx7
    Community Member
    Options

    Appreciate you taking a look. Here is the command and the error messages. Thanks.
    ... Mark

    MSL-MacBook-Pro-Late-2013:mrc-converter-suite Mark$ perl convert.pl msecure5 -v mSec6.csv
    Can't call method "year" on an undefined value at
    /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 477 (#1)
    (F) You used the syntax of a method call, but the slot filled by the
    object reference or package name contains an undefined value. Something
    like this will reproduce the error:

        $BADREF = undef;
        process $BADREF 1,2,3;
        $BADREF->process(1,2,3);
    

    Uncaught exception from user code:
    Can't call method "year" on an undefined value at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 477.
    Converters::Msecure5::date2monthYear("Expiration Date|11|11/2024") called at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 151
    Converters::Msecure5::ANON("Expiration Date|11|11/2024", "expiry") called at /Users/Mark/Downloads/mrc-converter-suite/./MCS/Core.pm line 151
    MCS::Core::normalize_records(MCS::Core=HASH(0x7fb241232440), MCS::Generators::PUX=HASH(0x7fb2404985d8), "creditcard", ARRAY(0x7fb24029e780), HASH(0x7fb24029e768)) called at /Users/Mark/Downloads/mrc-converter-suite/./Converters/Msecure5.pm line 379
    Converters::Msecure5::import(Converters::Msecure5=HASH(0x7fb240826da0), "mSec6.csv", undef) called at convert.pl line 200
    MSL-MacBook-Pro-Late-2013:mrc-converter-suite Mark$

  • MrC
    MrC
    Volunteer Moderator
    edited February 2023
    Options

    @netefx7

    Ok, this is an easy one to spot.

    The converter is trying to convert the credit card's month / year expiration value, but it is seeing the entire string "Expiration Date|11|11/2024" as being that month / year. Since the mSecure CSV export dos not contain any labels, your data must have that label / value as the date data. And clearly that's isn't a "date", let alone a month / year value.

    Would this data have been imported by mSecure from some other password manager, or did you add that? I thought I recall that mSecure5 validated date input, but perhaps it doesn't on importing from another password manager.

    Either way, I recently updated some of the date conversion routines to better handle such egregious data (normally I like to know when a password manager can create bogus data so that I get reports like yours).

    I just updated the development (not release) version of the suite. Can you retrieve that and give it a try? It's in the folder mrc-converter-suite - development version at top level of the download site. Follow the mrc-converter-suite home link here:

    https://1password.community/discussion/101693/moving-to-1password-from-another-password-manager

    Close your terminal window, toss your existing mrc-converter-suite into the trash, replace it with the unzipped development version I just posted, launch your terminal again, change directories and retry the conversion.

    Here is my run with your bad month / year value:

    $ perl convert.pl msecure5 /tmp/msecure5_mac.csv -v
    Import summary:
       11 items discovered
    Export summary:
        1 Bank Account item
        1 Credit Card item
        1 Email Account item
        1 Reward Program item
        2 Login items
        2 Membership items
        3 Secure Note items
       11 total items
    You may now import the file below into 1Password 8:
        /Users/cappella/Desktop/MCS_converted.1pux
    
  • netefx7
    netefx7
    Community Member
    Options

    Excellent, I'd be happy to give it a try. The csv file is as exported by mSecure 6. There are no controls provided to alter the export. I haven't tried importing to any other. I'm a bit perturbed that mSecure 6 exports in this fashion, but they may have their reasons. Doesn't make it easy, however.

  • netefx7
    netefx7
    Community Member
    Options

    Hmm. That went a lot better ...

    MSL-MacBook-Pro-Late-2013:mrc-converter-suite2 Mark$ perl convert.pl msecure5 mSec6.csv -v
    Use of uninitialized value $val in concatenation (.) or string at
    /Users/Mark/Downloads/mrc-converter-suite2/./Converters/Msecure5.pm line 352 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    and the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.
    

    Import summary:
    905 items discovered
    Export summary:
    3 Bank Account items
    3 Membership items
    3 Reward Program items
    15 Credit Card items
    311 Secure Note items
    570 Login items
    905 total items
    You may now import the file below into 1Password 8:
    /Users/Mark/Desktop/MCS_converted.1pux
    MSL-MacBook-Pro-Late-2013:mrc-converter-suite2 Mark$

    So I am successfully able to create the 1pux file. Nice, this is progress!

    However, I go to import into a new Test vault and can't seem to import the new file (see snap below). When I select import from the desktop application it only allows me to import directly from Lastpass. This never worked for unknown reasons -- I got it to work from the web interface.

    It looks like I should be able to feed it a file but the pop-up never allows me to select the "From File" tab. Hmm, this looks broken to me. Any suggestions?

    !

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    Yes, that is progress.

    The error might be innocuous. It's from a debug message. If the import works, then there's no need to worry about it.

    The import dialog where you pick LastPass or From File is goofy. Click on LastPass and it will switch. Then you can drag your 1PUX onto the drag area.

  • netefx7
    netefx7
    Community Member
    Options

    Right, click Lastpass to import from file -- a little counter-intuitive but you are right -- this allows me to import into a new vault called "Imported M_C_S". Cool, that works.

    The real problem is with the conversion of mSecure 6 data. Since the mSecure 6 export format seems to label|field#||value pairing for everything. The imported value for some usernames becomes "Username|7|testuser" for example. Passwords imported look like "Password|8|BogusPassword" which is not what was intended. The label such as "Password|8|" needs to be removed and just the remaining part of the string kept for the password.

    Similarly, entry names are formatted as title|ID for example "Cool Website|0123456789" which gets imported as a tag value. The title doesn't get used as the name of the 1password entry. Major bummer.

    I can't think of any reasonable way to fix this. I'd sure love it if your conversion tool handled this right! I believe this impacts everyone coming from mSecure 6 to 1password. Thanks.

    ... Mark

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    Oh, that's not good. Their export format has changed. I'll have to take a look. Can I get back to you tomorrow on this? I currently don't have a valid account and their software won't export without one (so I can't create test data).

  • netefx7
    netefx7
    Community Member
    Options

    Thanks for your help. I'm happy to assist with sample data and all. Tomorrow is good.

  • MrC
    MrC
    Volunteer Moderator
    edited February 2023
    Options

    @netefx7

    FYI: The mSecure 6 export format requires a completely new converter module (msecure6) to accommodate the large changes in the export format. I'm working on this now.

    Edit: Question, have you customized / changed any of your items stock field labels?

  • MrC
    MrC
    Volunteer Moderator
    Options

    Progress...

  • netefx7
    netefx7
    Community Member
    Options

    Thank you for working this! The main entry types/labels that I use are: Login, Computer Login, Credit Card, Serial Number, Secure Note, Account, Prescription, Combination, Frequent Flyer, Bank Accts, Frequent Flyer Hotel, Email Accts.

    Somehow along the way there were duplicates created like: Web Logins Duplicate, Credit Card Duplicate, Combinations Duplicate. I believe they are similarly but the fields move around a bit. Other than this, I didn't modify the structure at all just used it.

    This doesn't have to be a 100% solution but something get close would be most helpful. Thanks again.

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    In order to get you a converter as quickly as possible, I'll focus on what you need for your data, and will flush out the converter later.

    I'm working first on the stock Templates (and these will map to 1Password "Categories"). I've completed Login, Credit Card, Secure Note, Bank Account. I'll now add the other stock templates you list (Prescription, Combination, Frequent Flyer, Email Account).

    For your custom templates, I'll need to know how you want them mapped into 1Password's categories (without this, they will end up as Secure Notes and many will anyway). Your custom templates are: Computer Login, Serial Number, Account, Frequent Flyer Hotel.

    The converter currently is matching the field labels within each item to map these into 1Password's field's for a category. So if you've changed those labels, we'll have to add the appropriate pattern into the converter's field label pattern match definition. You'll know this when you examine the results, and we can adjust accordingly.

  • MrC
    MrC
    Volunteer Moderator
    Options

    In the Email Account template, what is the difference between these two fields?

  • MrC
    MrC
    Volunteer Moderator
    Options

    Is the "Name" field in Prescription a person's name, or a medication name?

  • netefx7
    netefx7
    Community Member
    Options

    Sound like a great approach. To answer your question about the custom templates. I'd suggest this mapping of custom template to 1password category

    Computer Login --> Login
    Serial Number --> Software License
    Account -- Drop this as too vague (fixed my one entry)
    Frequent Flyer Hotel --> Rewards Program

    Questions:

    Any 1password category for Insurance Info? (included snap of mSec6 types)

    We can iterate to polish this up for general usage. Thanks!

  • MrC
    MrC
    Volunteer Moderator
    edited February 2023
    Options

    @netefx7

    OK on the mappings. I need to know the exact field names of those templates (case sensitive).

    Regarding Insurance info, I have most converters map these into Membership, and have the converter add custom fields for things like Group #, Policy #, etc. In fact, I'm already handling this:

    So, as soon as you provide me with the field labels for each of those custom templates, we can start testing your new conversion.

    Also, see my other questions above too.

  • netefx7
    netefx7
    Community Member
    Options

    Sorry, missed I some of your email while I was look at this ...

    When I enter a new Email account, it presents the SMTP host field. I don't see the Outgoing Mail Server field. Perhaps it was deprecated. It is equivalent.

    The Prescription Name field is the medication name.

    Computer Login:
    Userid
    Password
    Custom 4
    Custom 5
    Custom 6

    Serial Numbers:
    Serial #
    Purchase Date
    Reseller
    Field 5
    Field 6

    Frequent Flyer Hotel
    Number
    Password
    URL
    Custom 5
    Custom 6

  • MrC
    MrC
    Volunteer Moderator
    edited February 2023
    Options

    @netefx7

    Sorry for the delay. I wanted to test on both Windows and macOS, since the CSV export differs based on platform.

    The Email account mapping may be wonky. The 1Password Email Account template was built when POP email and dial-up ISP access was common. If you don't like the mappings, let me know.

    I've updated the dev. release so you can test. The converter's name is msecure6. So use it as in:

    perl convert.pl msecure6 -v ..\msecure_export.csv
    

    You can add the --debug (or -d) option if you want to review the mappings without importing into 1Password.

    Some notes on your custom templates:

    1. I am not mapping the Custom # fields; they will go into the Notes area. If you want them mapped, let me know.

    2. The Serial # and Reseller fields for your Serial Numbers template does not map naturally to any fields in the Software License category, so they will become custom fields. I don't know what data type your Purchase Date is, and I've presumed it is a Date Field type (which has a date picker and ID value 10 in the CSV export (the middle value in the pipe-separated value, as in label|id|value). This is important because the date value is in a specific format that gets converted to 1Password's internal representation for date values for date fields. If your Purchase Date is just text, I'll redefine that definition to not attempt to convert this value.

    3. I have not created custom templates identical to yours, so haven't tested with any data - you'll have give me feedback on the mapping.

    I'll await your feedback.

    Edit: I updated the dev version to resolve an issue with placing empty string tags in each item.

    Edit: I updated the dev version so that the README is now updated with this new msecure6 converter.

  • netefx7
    netefx7
    Community Member
    Options

    Excellent. I'll give it a shot on Thursday. Thanks!

  • netefx7
    netefx7
    Community Member
    Options

    Conversion went pretty well ...

    MSLMBPLate2013:mrc-converter-suite-2B Mark$ perl convert.pl msecure6 -v ./mSecure6ImportA.csv
    Import summary:
    817 items discovered
    1 items were added by splitting 1 original item
    Export summary:
    1 Medical Record item
    2 Software License items
    3 Bank Account items
    3 Membership items
    7 Reward Program items
    13 Credit Card items
    206 Secure Note items
    583 Login items
    818 total items
    You may now import the file below into 1Password 8:
    /Users/Mark/Desktop/MCS_converted.1pux

    Loaded into 1password and looked pretty reasonable. Didn't look bad at all. Looks like a lot of things turned into Secure Notes, however. Could it be that all the Duplicate types turned into Secure Notes? I'll get back to you on Friday with a better analysis of what happened.

    Once something gets categorized, can I edit it to change the categorization? I can do this in mSecure. Thanks!

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    Good progress!

    Categories cannot be changed, so its worth getting this right w/the converter. Examine a few big hit categories that you feel aren't correct, and we can adjust to hit the majority.

    Dup's don't cause incorrect categorization - the category is defined in the CSV, and the converter maps that to a 1Password category where it can. Some mSecure templates don't map naturally, so they may end up as Secure Notes.

  • MrC
    MrC
    Volunteer Moderator
    edited February 2023
    Options

    @netefx7

    I've updated the dev version of the suite to support all of the stock mSecure 6 templates:

    The templates added are: Router Settings, Software License, Vehicle Info, and Voice Mail.

  • netefx7
    netefx7
    Community Member
    Options

    Another pass on this and it looks pretty good. Here is the output ...

    MSL-MacBook-Pro-Late-2013:mrc-converter-suite-2C Mark$ perl convert.pl msecure6 -v ./mSecure6ImportD.csv
    Import summary:
    817 items discovered
    1 item was added by splitting 1 original item
    Export summary:
    1 Email Account item
    1 Medical Record item
    1 Passport item
    2 Software License items
    4 Membership items
    5 Bank Account items
    13 Reward Program items
    15 Credit Card items
    49 Secure Note items
    727 Login items
    818 total items
    You may now import the file below into 1Password 8:
    /Users/Mark/Desktop/MCS_converted.1pux

    I may do some more converting but I think I'm close enough. Now this becomes a manual process of cleaning everything up. I'm now able to see all entries from Lastpass and mSecure6 together in 1password.

    MRC -- Thanks so much for jumping on this and providing the conversion tool. You made this process doable and saved me hours! I really appreciate it. Other mSecure 6 guys to come will thank you too.

    All the best.

  • MrC
    MrC
    Volunteer Moderator
    Options

    @netefx7

    You're very welcome. Those results do look better. Thanks for all your help on this.

    I'll keep your customized definitions around for a bit, in case you want a little more fine tuning.

    And I'll make an official update to the suite this weekend.

    Enjoy 1Password!

This discussion has been closed.