General question about autofill failures

Quantumpanda

I have been aware of issues with 1P filling in the wrong fields on some sites for a long time, but I had not bothered to report anything because I was always trying to get something actually done, and manually copying and pasting info took care of the problem for that moment. Until today, I wasn't aware of the "Report Website Issue" feature, but now I will definitely start using it to report specific issues.

My question is more general. The most common situation I've seen 1P fill in the wrong fields for (not the only case by a long shot, just the most common recurring issue) is when a site's login page contains username/password fields for both logging in and creating a new account. 1P always seems to choose the new account fields rather than the login fields. This behavior is consistent across multiple websites.

A prominent site this can be tested on is https://linkedin.com/. If you log out from LinkedIn then visit their homepage (not the "Sign In" link given when you log out), you will see that in the center of the page there is a box for creating a new account which includes email and password fields, but there is also a pair of regular sign in fields at the top of the page. (And yes, I'm going to report the specific site after I finish this post. I'm using it as an example here.) This is typical of the kind of pages I'm talking about.

So my question is: might there be a way to fix this particular issue within 1P in general, rather than reporting each website individually? For example, if 1P finds more than one field on a page labeled "password", it might ask the user which one to fill the first time, and then store that for the future? The page code should contain something that distinguishes those fields internally.

Similarly, when 1P saves a new login entry from a site's "new account" page, this tends to mess up the autofill fields as well. In these cases, 1P stores the account creation page as the go-to page for the site (which IMO should never happen, since you're not likely to need to go back to that particular page again), and while it recognizes the domain and offers the correct login info, it frequently can't match the fields because the fields on the account creation page and those on the sign in page often aren't labeled or identified the same. When I find one of these cases, I have to go into the saved login and edit the stored URL, and then I usually have to do the manually copy-and-paste thing again.

As with the other issue, might there be a way for 1P to recognize that it's pulling info from an account creation page, and modify its behavior accordingly? I'm not sure what might be the best default behavior.

AGAlumB

So my question is: might there be a way to fix this particular issue within 1P in general, rather than reporting each website individually? For example, if 1P finds more than one field on a page labeled "password", it might ask the user which one to fill the first time, and then store that for the future? The page code should contain something that distinguishes those fields internally.

@Quantumpanda: Obviously, this will vary from site to site. But 9 out of 10 times, if 1Password is filling the 'wrong' fields (or both sets), it's because the IDs are the same...or were when your Login item was saved initially — probably during the signup process. In most cases it isn't possible currently for 1Password to programmatically differentiate between two 'password' fields on the same page that are identical.

Similarly, when 1P saves a new login entry from a site's "new account" page, this tends to mess up the autofill fields as well. In these cases, 1P stores the account creation page as the go-to page for the site (which IMO should never happen, since you're not likely to need to go back to that particular page again), and while it recognizes the domain and offers the correct login info, it frequently can't match the fields because the fields on the account creation page and those on the sign in page often aren't labeled or identified the same. When I find one of these cases, I have to go into the saved login and edit the stored URL, and then I usually have to do the manually copy-and-paste thing again.

If this is where you saved the login, how is 1Password to know the 'correct' URL and form details of a page it knows nothing about? Computer hardware and software only 'know' the information we give them. It just isn't possible (yet?!) for 1Password or anything else to intuit intention and have the awareness to anticipate what we want it to do and the agency to make it happen without us explicitly telling it to. This, of course, is the Holy Grail of Technology. I suspect someday we will get there, and it will be AMAZING...right until the machines take over. Enjoy it while it lasts! :sunglasses:

But all joking aside, in the case of Linkedin, I was able to manually save a login and fill it without encountering the problem you describe...so I suspect 1Password simply has outdated information about the form in a an old Login item you saved. You can do this too:

1. Navigate to the website
2. Enter your login credentials
3. Tap the 'keyhole' icon to bring up the extension
4. Click the 'gear' icon
5. Click Save New Login
6. Give it a name and Save
7. Close the webpage and select your new login from the extension to have 1Password Go & Fill
8. Submit the form manually if you have autosumbit disabled

And while it doesn't appear to be necessary in this particular case, there's also another good example on Linkedin. In some cases if the general homepage/loginform/signupform is problematic, simply using another more purpose-built login page can help:

https://www.linkedin.com/uas/login-submit

And finally, when all else fails, do exactly what you've been doing! Post here, let us know the URL, your setup, and the situation, and we'll do what we can to help. Ultimately though it is a game of whack-a-mole, because all login forms are different. And often a tweak that will help in one case will cause problems in another. But we're not giving up. ;)

Quantumpanda

@brenty, thanks for the response. It pretty much confirmed what I suspected, but it never hurts to ask.

That said, I wanted to respond to a couple of things you mentioned here:

If this is where you saved the login, how is 1Password to know the 'correct' URL and form details of a page it knows nothing about?

I guess I wasn't as clear as I could have been. I wasn't asking to have 1P figure out the correct info by itself—I was asking if there was a way it could detect that this page might be an account creation page (based on the fields on the page—for example, most new account pages ask you to enter the password twice to confirm it, but most login pages don't) and ask the user if this is the page it should save.

Now, based on your response, I can tell that this isn't as simple as it might seem, but I knew that already. Every page is constructed just differently enough to prevent there from being any perfect method of detecting anything about any arbitrary page. Ultimately, it's the user who has to determine what's correct.

One of the reasons I find this issue annoying is that saving a new login from the browser is all or nothing. If you have 1P create a new login from the "save login" window, it saves all the information it can guess, without the user having the opportunity to confirm or edit anything. If you opt to not have it save it, then it saves nothing unless you had it generate a password, in which case it still saves the password and URL as a record that can be converted later. That conversion process itself is less than optimum, since when you convert a generated password into a login entry, it doesn't have the username—and it doesn't prompt the user to add it. And it's still the same wrong URL that it would have been if you'd had 1P save the login in the first place.

In the end, these all boil down to the fact that no matter what I do, anytime I create a new account on a web site, I will have to go back at some point and manually create the login entry, visit the site's login page to save it correctly, or edit the one 1P created if I want the info in it to be correct. This is a lot of extra work for a utility that claims to manage all that for me.

Maybe the best way to address this would be to have the "save login" window include the option to edit the info at the time you save the new entry. Then I could fix the URL immediately and correct field matching if necessary. Similarly, when converting a generated password to a login entry, 1P could highlight the empty username field so that the user knows it needs to be filled in. Maybe include a checkbox to confirm that no username should be stored with this password (unlikely with web sites, but not unusual with some other kinds of passwords, such as encrypted disks), so that 1P doesn't keep bugging the user for it if there's not going to be one.

Ultimately, what this comes down to is giving the user easy ways to address the things that 1P can't figure out itself. That's the same philosophy behind the in-app issue reporting: it makes it easy to report a site that's causing problems without having to keep coming back to these forums and posting messages about them. I just want it to be similarly easy to make sure new login entries are correct.

in the case of Linkedin, I was able to manually save a login and fill it without encountering the problem you describe...so I suspect 1Password simply has outdated information about the form in a an old Login item you saved.

So why can't 1P detect that the form on the page doesn't match that of the page it saved the entry from, and prompt the user to verify that it's filling the info correctly? This is a case where 1P actually is better equipped to figure out that something's different than the user is: 1P saves internal details about the pages it creates login entries from, and can readily compare them to the page you're asking it to fill in. How is the user supposed to know that the info 1P has is outdated? Most users couldn't even tell you if the page they logged in on today looked any different from the one they logged in on yesterday, much less figure out if any internal structure has changed. All we see is 1P misfilling the form, and it won't occur to most people to think that the problem comes from the web page having changed since they saved their login. And as often as some web sites change their login pages, we're again looking at a lot of manual work for the user that we expect the utility to help us with.

Okay, I get it that this is not an easy problem to address. But you sell 1P, in part, on the claims that it makes managing your passwords easy. To most people, that's going to mean that they can save a password in it and forget about it, and 1P will tell them if they need to do something different to it. I understand that giving 1P the intelligence to fix these problems itself is at best impractical—but can't it at least have enough brains to tell us that something's wrong and walk us through (or even automate) the process of fixing it? Is that too much to ask?

I get the impression from several messages on these forums that the most common fix for field-filling problems is to re-save the login entry. If that's such a common process, why do users only find out that it's needed by coming here and asking? You could make the first step of the "Report Website Issue" process be a popup that advises the user to try that resaving process first. You could even make it smart enough to remember what page you were on when you selected the report command, and if you re-select the command from that same page within a reasonably short period of time, it skips the popup and goes straight to the reporting system. That simple addition would probably greatly reduce the number of "website issues" reported that are issues only because the user's login entry needs resaved, and that would mean more time for your team to do something other than track down reported website issues.

Sometimes, it seems like your dev team puts so much focus on making sure that 1P stays secure (which I'll grant is very important) that making it better at helping the user manage their passwords is given short shrift. If security were all that mattered, I'd save my money and just put my passwords in an encrypted text file on an encrypted disk image and keep that in my DropBox. I need my password manager to make it easier to manage my passwords (and other secure info). You've done a great job with the Security Audit features, that can proactively show the user which passwords may be at risk or insecure. You need to find ways to extend that into other aspects of password management, apply that kind of intelligence and initiative to keeping the myriad entries up to date with the changes to the associated sites.

littlebobbytables

Greetings @Quantumpanda,

You make a great point with the idea of proactively suggesting the creation of a new Login item if a person is indicating they want to report a problem filling. That entire piece of functionality is quite new and rough around the edges and your idea is definitely one that has merit. In fact I've just finished writing up the entry in our tracker for it. We can have it recommend a particular KB article only if the issue is with a Login item too. I also found reporting a website issue was leaving 1Password mini open so I've reported that too. So thank you, we got a two for one deal with your post :sunglasses:

So now to try and respond to some of your other thoughts. Currently what 1Password should do is not ask a person if they want to save a Login item if we detect that they're on a website's registration page. As you've already noted, most of this is based on educated observations of typical approaches. If we see only two password fields fields containing the same entry we assume it's a registration page (most password change pages ask for the old password too). We also look for certain keywords in the path. If we detect it's a registration page we don't want to save because it almost undoubtedly contains information that isn't needed to log in and more importantly the page will likely have zero relation to the actual login page.

Now password change pages are different. We don't want to record that particular page but we do want to update an existing Login item. If we see an old password and a new one twice we interpret it differently. When we update a Login item all we do is change the value of the password field in the belief that nothing else needs touched. We have had a bug recently that should be resolved in the latest beta where it wasn't always asking when it should in those cases.

Here is how I see getting the best usage out of 1Password at the moment and is pretty much the steps I take any time I create a new Login item.

1. You register with a new site and at some point in the process you have 1Password's Password Generator generate a new password for you on their registration page. 1Password will save that password in a Password item that will also record the domain and title of the site to aid in identification.
2. You then log out of the site if part of the registration process logged you in automatically.
3. You visit the site's login page and fill in your username. You use the Password item to fill in the password.
4. 1Password should ask if you want to save a new Login item. Seeing that you're using a password that exists in a Password item it will convert that Password item into a Login item and update any required fields like the username field and specific URL etc.

Software development is always an ongoing process. The software continues to evolve as does our documentation. While there are moments while our developers are happy with something they've done it's merely a pause before they turn their attention to the next task and an ongoing battle not just here but in any software development is that very fine balance between the desire to perfect one piece of functionality and the need to address all areas and get releases out there for the users to benefit from. Basically I doubt there will never be a time where we say something can't be improved is what I'm trying to say. I spend a bit of time in the extension and my problem isn't coming up with ideas on what might improve the UI but how realistic some of the ideas are (are they actually feasible?) and securing enough time to see it happen, especially as the extension relies on 1Password and this can mean dev time on both OS X and Windows depending on the change. I believe you're quite right though, there are ample opportunities to make this nicer and I believe with time it will happen.

You're correct, securing your data is critical but while there are times that a particular usability suggestion will conflict often they are independent and the UI is important. From a psychological viewpoint a slick looking application has a certain importance but a well designed UI is just a pleasure to use.

Regarding your comment about if a web page changes. Again, you're completely right. If a web page makes a bunch of alterations to the underlying structure e.g. a refactoring alters the IDs the user can't see any difference without the dangerous move of viewing the source. All I will say on that point is careful, that way can lead to madness :tongue: With a certain level of caution maybe there are things we can do to suggest if this is the case. We do have to be careful though. For example we did try looking into an automated approach for identifying if filling failed but the issue was it was incredibly unreliable due to timings and the false positives took what would have been a great idea and consigned it back to the wish list.

I would continue this engaging discussion but I know we have a number of users awaiting replies at the moment so I'll pause for now and see what your thoughts on my thoughts are :smile:

ref: AGW-260

Quantumpanda

Thanks, @littlebobbytables, for the extensive reply. It's the rare developer that's willing to carry on this level of discussion with a user. I'll apologize if any of my comments have sounded frustrated or exasperated—I'm so used to getting the TL;DR response that I sometimes preemptively expect it and word things accordingly. I know I'm wordy, but with these kinds of issues that's mainly because I have a lot to say. I think too much about some of this stuff. :-)

As for specifics...

Here is how I see getting the best usage out of 1Password at the moment and is pretty much the steps I take any time I create a new Login item.

1. You register with a new site and at some point in the process you have 1Password's Password Generator generate a new password for you on their registration page. 1Password will save that password in a Password item that will also record the domain and title of the site to aid in identification.
2. You then log out of the site if part of the registration process logged you in automatically.
3. You visit the site's login page and fill in your username. You use the Password item to fill in the password.
4. 1Password should ask if you want to save a new Login item. Seeing that you're using a password that exists in a Password item it will convert that Password item into a Login item and update any required fields like the username field and specific URL etc.

And here is where I think my issue comes in. I don't normally log out immediately after creating a new account somewhere, and I don't normally immediately follow that with a login to get 1P to save it. Doing it that way has literally never occurred to me. I always use the "Convert to login" button in the extension to create the login, and I normally do it right after creating the account, so that I don't forget to create the full login entry. And doing it that way results in a login entry with no username.

I'm definitely going to try it your way the next time. But now I can see why we're seeing this differently. I'm using the method that to me seems obvious: the panel showing the password I had just generated and used has a button in it labeled as "this is how you do this". This is how I generally deal with a UI; I look for buttons and menu commands to do things, and if I find the commands, I will normally prefer to initiate these actions myself rather than wait for an automated process to do it, unless I have been given reason to trust that the app will actually do it. (I've been using computers for far too long to automatically trust a program to do anything it says it will. A computer is only as smart as the people who programmed it.)

Also, to me, having to log out and then log back in to get 1P to save the login entry properly seems like an extra step I shouldn't have to take. I understand the explanations I'm getting here for why that seems to be the best way currently available to do it, but it runs counter to my expectations for a program that's supposed to manage my passwords "automatically". Maybe I'm the only one who thinks this way. Somehow I doubt it, but it's possible. In either case, it seems to me that if you have a button, clearly displayed, to convert a generated password to a login entry, that button should work as well as any other recommended process of creating that entry.

Regarding your comment about if a web page changes. Again, you're completely right. If a web page makes a bunch of alterations to the underlying structure e.g. a refactoring alters the IDs the user can't see any difference without the dangerous move of viewing the source. All I will say on that point is careful, that way can lead to madness :tongue:

Too late, I'm already there. :-)

With a certain level of caution maybe there are things we can do to suggest if this is the case. We do have to be careful though. For example we did try looking into an automated approach for identifying if filling failed but the issue was it was incredibly unreliable due to timings and the false positives took what would have been a great idea and consigned it back to the wish list.

Which brings us back around to the fact that there are just too many different ways to do some things in web sites, and instead of settling on a few standards that work really well, an enormous chunk of web designers and coders prefer to keep reinventing the wheel and rolling their own way of doing things. Even if everyone was producing perfect code (and not only are most not doing so, too many aren't even checking to see if their code validates at all—and code generated by some web design software is so absolutely atrocious it's a wonder it renders at all), there still wouldn't be any reliable way to know what's going on under the hood 100.00% of the time.

As I'm thinking about the issues related to web sites that keep changing their code, an idea has come to mind. One of my more recent annoyances in this field is the growing prevalence of two-step logins (not to be confused with two-factor authentication), where you enter your username on one page, submit, and then enter your password on a second page. There still isn't really any graceful way for 1P to handle those. If the login entry was already created, I can simply fill it on both pages and it works, because each page has only one of the two key fields. Otherwise, there's no simple way for 1P to do it. You can create separate entries for username and password, or you can have 1P auto-save an entry for one and manually add the other, or you can just manually create the entire entry, but these are all suboptimal solutions.

Would it be possible for 1P to compare the info on successive pages, so that if you type only a username on one page, and then within the next few pages you type only a password (you can't assume it will be only the next page because interstitial pages are common and sometimes unpredictable), 1P suggests combining the two into a single login entry somehow? I'm not sure what the best approach for this would be—I can think of a few different ways to do it—but if 1P could automatically recognize these two-step logins, that would be a really nice feature.

AGAlumB

Thanks, @littlebobbytables, for the extensive reply. It's the rare developer that's willing to carry on this level of discussion with a user. I'll apologize if any of my comments have sounded frustrated or exasperated—I'm so used to getting the TL;DR response that I sometimes preemptively expect it and word things accordingly. I know I'm wordy, but with these kinds of issues that's mainly because I have a lot to say. I think too much about some of this stuff. :-)

@Quantumpanda: No need for apology! Thank you for the thoughtful and honest feedback. There's certainly no offense taken, and if I come across as exasperated or frustrated I think we're coming at this from the same place: we want 1Password to be better! :chuffed:

Now, based on your response, I can tell that this isn't as simple as it might seem, but I knew that already. Every page is constructed just differently enough to prevent there from being any perfect method of detecting anything about any arbitrary page. Ultimately, it's the user who has to determine what's correct.

@Quantumpanda: I think this sums it up nicely, but I do have one thing to add: our priority is login forms, rather than signup forms. Now, if in the future we have 'cracked' login filling and 1Password is capable of filling virtually all login forms with ease, then we'll move on to a new challenge — perhaps this signup issue you raise, which could certainly be a good 'quality of life' improvement. But I wonder if we'll ever see the day. :lol:

In the end, these all boil down to the fact that no matter what I do, anytime I create a new account on a web site, I will have to go back at some point and manually create the login entry, visit the site's login page to save it correctly, or edit the one 1P created if I want the info in it to be correct. This is a lot of extra work for a utility that claims to manage all that for me.

It's definitely an interesting idea, but I think this just shifts the 'user work' part from editing a Login item to clean it up to prompting the user to clean it up before saving. It certainly serves as a reminder, but I suspect many people would find this rather annoying — myself included.

Personally, I like being able to capture all of this data quickly and then edit it later if needed. In my experience, editing isn't often necessary; but because I am a bit particular, I tend to edit items more than needed, just to streamline things, especially for Logins I use frequently.

1Password isn't meant to do all of this work for you. After all, that would entail having it make decisions on your behalf about what to keep and what to throw out that may be overstepping. And your suggestions, while I can see how they would appeal to some people, would frustrate and confuse others. And as I said earlier, ultimately having the autosave window prompt you to decide what to save and what not to doesn't remove any of the work; it merely changes the work flow — of course, in a way that appeals to you, which is why you're here! ;)

Also, to me, having to log out and then log back in to get 1P to save the login entry properly seems like an extra step I shouldn't have to take.

This is (maybe) going to blow your mind: Most people don't take this step, and 1Password still works for them! :eh:

And here is where I think my issue comes in. I don't normally log out immediately after creating a new account somewhere, and I don't normally immediately follow that with a login to get 1P to save it. Doing it that way has literally never occurred to me. I always use the "Convert to login" button in the extension to create the login, and I normally do it right after creating the account, so that I don't forget to create the full login entry. And doing it that way results in a login entry with no username.

This is pure gold! Why, you ask? Because that makes at least three different workflows for saving a new Login (at least off the top of my head): 1 yours, 2 lil bobby's (which is similar enough to mine), and 3 (I think) almost everybody else's, which is as follows:

1. Signup for an account with a website
2. Save it in 1Password when prompted
3. Try to login to the same site later — "Whats my password?"
4. "I bet it's in 1Password!" — open the 1Password extension/mini...
5. 1Password fills the username and password!

I think what's easy for you and I to miss as more advanced users is that 1Password is an afterthought. It's a contingency plan for those of us who don't want to spend any time thinking about this. We're not in that group, because we're sitting here having a philosophical discussion about it. And when that user has that "Eureka!" moment when they realize that 1Password already has what they need, all they do is have it fill the login form they are already at.

Now, having saved a Login item previously at the 'signup' page may result in it being different enough that it won't fill the actual login form...and of course that's when we hear from them. However, that problem is relatively rare. You may not think so, since these forums are filled with people with login issues, but of course this is self-selecting for those who are having trouble in the first place. That's not to say that it isn't a problem we'd like to solve (if such a thing can be), but frankly we wouldn't be able to keep up with the volume if it was more widespread!

Hopefully that description is adequate, because I'm coming back around to my earlier point, which depends on it: Group 3 Users (a designation I just made up) are the ones who will be the most annoyed or baffled by 1Password asking them to make decisions about URLs and form data when saving a new Login item. Many would prefer (or believe) that 1Password doesn't prompt them at all, and just autosaves in the background. So if we were to implement the sorts of changes you're suggesting in the future, a lot of other things would have to change too. And I apologize for sounding so pessimistic, but I prefer to temper my expectations so that I can be delighted "when a plan comes together".

I understand the explanations I'm getting here for why that seems to be the best way currently available to do it, but it runs counter to my expectations for a program that's supposed to manage my passwords "automatically".

Many people think this way, but in my experience the person expecting 1Password to do everything automatically is not usually the same person who writes this:

(I've been using computers for far too long to automatically trust a program to do anything it says it will. A computer is only as smart as the people who programmed it.)

I have a similar philosophy, but most people don't take this into consideration. :crazy:

Which brings us back around to the fact that there are just too many different ways to do some things in web sites, and instead of settling on a few standards that work really well, an enormous chunk of web designers and coders prefer to keep reinventing the wheel and rolling their own way of doing things.

Okay, now you're just preaching to the choir. Can I get an "Amen!"

As I'm thinking about the issues related to web sites that keep changing their code, an idea has come to mind. One of my more recent annoyances in this field is the growing prevalence of two-step logins (not to be confused with two-factor authentication),

Very timely, since there's been some discussion about the multi-page login issue and TOTP. Google has these each on separate pages, and it looks like Amazon is going that route too. Very frustrating, no matter who you are!

I'm not sure what the best approach for this would be—I can think of a few different ways to do it—but if 1P could automatically recognize these two-step logins, that would be a really nice feature.

I agree wholeheartedly! Rather than going off on a pessimistic rant, though, I'll say that because Amazon and Google are so...prolific(? ubiquitous?) I wonder if we could in the future leverage their own 'standardization' of their login forms to help streamline 1Password login filling with them. Just a thought.

Your feedback is awesome, and I think you have a unique perspective. Even if it turns out that some of these are pipe dreams for technical, social, or resource reasons, it gets us thinking about things from different angles. Thanks so much for taking the time (and energy!) to share your thoughts and experiences. :pirate: