Wildcard hostname matching

Pilar

Hi @slvrstn

Thanks for your vote, we'll for sure add it ;) Happy New Year!

n8tr

Come on guys, this was requested 2 yrs ago. Why in the world is this not done by now?

Drew_AG

Hi @n8tr,

Can you please be more specific about the request you're referring to? There are actually a few different things that have been discussed in this thread. Some customers have asked to support using an asterisk in a website URL as a wildcard so 1Password will fill login credentials on all subdomains of a certain site. But that shouldn't be necessary in many cases, because 1Password can already do that. For example, if you have a Login item for https://calendar.google.com, that Login will also work on other google.com subdomains (accounts.google.com, plus.google.com, etc). You might simply need to enable 'Allow filling on pages that closely match saved websites' in 1Password > Preferences.

However, there was a bug which was preventing 1Password from filling certain subdomains on some sites. Are you having a problem like that? If so, and if you don't mind letting us know the URL(s) where you're having trouble, we can take a look and see what's going on.

Thanks in advance! :)

ChrisTechSmith

Hi 1Password team! Love the software and I recommend it to everyone who could make use of your solution. I've tried several competitor products and always come back to 1Password. For this specific thread, I wanted to add my vote to getting wildcard matching on the Mac version to work.

I'd be willing to bet that the majority of folks sounding off on this thread are web developers. In today's cloud hosting environments it is pretty common to have on-demand "elastic" services that make NOT having wildcard support to website matching a real problem. Here are some specifics to my situation:

We use dynamic subdomains when working on a code branch so that code changes can be reviewed in isolation before they are merged back into our master repository. So for feature A we might have a URL for testing that looks like this: https://pp-foo-staging-pr-1234.appdomain.com and for feature B we might have a URL for testing that looks like this: https://pp-foo-staging-pr-1235.appdomain.com. I have the same user name and password for both URLs (it's the same web application) but 1Password presently doesn't recognize this and forces me to save each as individual accounts. 1st problem with this is that we spin up these dynamic subdomains 5 or 10 dimes in a day so in 1 weeks time I'd have potentially 50 new 1Password accounts to deal with. 2nd problem with this is that these dynamic subdomains are temporary so once the change request is merged the subdomain is killed off and I will NEVER reuse the account I was forced to save into 1Password.

I'm a 1Password for Mac user. In my Preferences I have "Allow filling on pages that closely match saved websites" checked on. I have added https://*.appdomain.com to the login account in 1Password. I also have several other iterations including .appdomain.com and appdomain.com. None of this helps for my specific use case and I'm stuck with having to have my user name and password in plain text in a text editor. (I've even resorted to trying to get TextExpander and Pastebot to pinch hit for me.) Adding this feature asap would be huge for me!

AGAlumB

I'm a 1Password for Mac user. In my Preferences I have "Allow filling on pages that closely match saved websites" checked on. I have added https://*.appdomain.com to the login account in 1Password.

@ChrisTechSmith: I guess I don't understand why you're adding a wildcard there, given that you're requesting that we add this feature in the future. In fact, I think you may just be making more work for yourself.

Given that https://pp-foo-staging-pr-1234.appdomain.com and https://pp-foo-staging-pr-1235.appdomain.com both share the same TLD, I'm not sure why you wouldn't just use a single login item with https://appdomain.com as the URL for both of these. Does that help?

We definitely want to improve how domain matching works in general, but from your description it sounds like this can work pretty close to the way you want it to now. Let me know what you find!

littlebobbytables

Hi @ChrisTechSmith,

I see a lot of confusion over how all of this currently works and not just from our users so I'd like to take a moment to expand on what Brenty has written.

Firstly we look at:

1. Does a Login item contain a match for the FQDN (Fully Qualified Domain Name) e.g. pp-foo-staging-pr-1234.appdomain.com
2. Does a Login item contain a match for the registered domain e.g. appdomain.com

This is simple string matching here, nothing more.

By default the Allow filling on pages that closely match saved websites is off and 1Password's behaviour is as follows.

1. If there is one or more matches to the FQDN we visibly display these in 1Password mini. If there are other matches to the registered domain we hide these behind the menu entry Show x more items.
2. If there are no matches to the FQDN we simply display any matches to the registered domain.

Our keyboard shortcut ⌘\ will fill automatically without any user input if the results of the above are only one visible Login item regardless of whether it's a FQDN or registered domain match. If 1Password mini displays two or more 1Password will always display the 1Password mini menu and ask you to choose.

What the Allow filling on pages that closely match saved websites option does is disable the FQDN check and everything is only looked at from the perspective of the registered domain.

So a single Login item with a website field set to https://appdomain.com should match to any page on any subdomain of appdomain.com if the TLD (Top-Level Domain) is a recognised one like com. It means 1Password will behave differently if you're creating you're own TLDs.

I've seen people believe that 1Password supports wildcards because 1Password seemed to work when they had https://*.appdomain.com stored in the website field but what was really happening was the string match for the FQDN failed but the registered domain match passed. We also see a lot of people enable the Allow filling on pages that closely match saved websites option but in most cases people don't really want it. In fact I'm not sure what real scenarios exist where the option is genuinely useful, you'd need multiple Login items over different subdomains and find it annoying to click the Show x more items each time to reach the item you want. My suspicion is after reading this you may agree and you disable the Allow filling on pages that closely match saved websites option as I don't believe it's doing what you hoped it did.

I wonder if the issue you're facing is more to do with the TLD. Assuming a single Login item with https://appdomain.com for the website field doesn't work, what do you use for the TLD?

Sorry about the confusion we've caused, the black box nature of the matching seems to have this effect.

ChrisTechSmith

brenty said: " I'm not sure why you wouldn't just use a single login item with https://appdomain.com as the URL for both of these. Does that help?"

No it doesn't help because it doesn't work.

littlebobbytables said: "I wonder if the issue you're facing is more to do with the TLD. Assuming a single Login item with https://appdomain.com for the website field doesn't work, what do you use for the TLD?"

I am guessing now that the problem is the TLD. I was using an example in my post above but the actual TLD is herokuapp.com. If you attempt to access the TLD directly you get rerouted to heroku.com.

Following your direction I removed my previously saved login item in 1Password and created a new one which looks like this:

username: [my@username]
password: *********************
website 1: https://herokuapp.com

1Password Mini in Safari does not show ANY logins when I land on a FQDN in the form of https://pp-foo-staging-pr-1236.herokuapp.com/. I have "Allow filling on pages that closely match saved websites" checked to ON. Is there something I'm missing or doing wrong?

AGAlumB

No it doesn't help because it doesn't work.

@ChrisTechSmith: It would according to your example. If it didn't, 1Password wouldn't be able to match any logins to websites. But this makes sense:

I am guessing now that the problem is the TLD. I was using an example in my post above but the actual TLD is herokuapp.com. If you attempt to access the TLD directly you get rerouted to heroku.com.

It sounds like you just need to add a heroku.com URL to the login item, so that 1Password can know you want it to be offered there. Does that help?

ChrisTechSmith

It sounds like you just need to add a heroku.com URL to the login item, so that 1Password can know you want it to be offered there. Does that help?

Thanks brenty. Unfortunately, no it didn't help. For website values, I now have the following
website 1: https://herokuapp.com
website 2: https://heroku.com
website 3: http://heroku.com (just because I wasn't sure if the SSL/non-SSL mattered.)

1Password Mini shows no logins for this item when I visit a site in the form of: https://pp-foo-staging-pr-1236.herokuapp.com/. If I visit heroku.com I get both my actuall Heroku login and also the one above. What I need though, is for 1Password Mini to locate my previous example on the temporary app URL but no love yet. Any other suggestions?

littlebobbytables

Greetings @ChrisTechSmith,

Can you do me a favour please. Can you visit https://pp-foo-staging-pr-1236.herokuapp.com/ and use the steps detailed on our page How to save a Login manually in your browser to create a new Login item. I'm curious to a couple of things.

1. Does 1Password correctly display this Login item?
2. What does 1Password have stored in the website field(s)?

I'm wondering if we learn anything about how 1Password is seeing the site from what it automatically stores in a new Login item. With a little luck it will highlight something that helps explain the odd behaviour so far.

ChrisTechSmith

Thanks littlebobbytables. I've done as you asked and here's what I have:

In the manually created Login item the value for website is "https://pp-foo-staging-pr-1260.herokuapp.com/". If I visit that URL again, I'm good. 1Password correctly displays the Login item. However, after one time use, I won't be going back to that exact URL again. The next item I need to access is located in a slightly different URL "https://pp-foo-staging-pr-1264.herokuapp.com/". 1Password does not show a Login item at all for that URL.

littlebobbytables

Hello @ChrisTechSmith,

Very strange. Can you try something for me please, you said you had the Allow filling on pages that closely match saved websites enabled and it sounds like it has been enabled throughout the entire issue. Can you disable it and see what impact it has.

jxpx777

Sorry for the confusion, @ChrisTechSmith. This is actually an expected behavior. We use Mozilla's public list of URL suffixes to determine the "root" domain of a URL. For something like 1Password.com, the suffix is just .com and 1password.com is the root domain for support.1password.com, myfamilyname.1password.com, etc. But for Heroku apps, each subdomain is a separate app, so herokuapp.com is the suffix. In this situation, 1Password reads pp-foo-staging-pr-1260.herokuapp.com as effectively a totally separate site called pp-foo-staging-pr-1260 in the herokuapp.com "TLD" just the same as 1Password.com is a site at 1password in the com TLD. (This isn't technically true, but it's a useful way to describe what 1Password is doing in this situation.) Unfortunately, there's not really a good way to get these to recognize. You can try setting your URL to just herokuapp.com and see if that helps, but off the top of my head I can't say if that will work for you. I'd love to hear if it does help, though!

--
Jamie Phelps
Code Wrangler @ AgileBits
Fort Worth, Texas

littlebobbytables

Hello @ChrisTechSmith,

I apologise, I had no idea it was that crazy. I clearly need to chat with Jamie and I'd love to learn why this was added to the public list of suffixes.

zupo

Hey @jxpx777 I have exactly the same use case as @ChrisTechSmith, and the same problem.

I have tried setting the URL to herokuapp.com, but it does not help. The URL has to be the full foo-pr-123.herokuapp.com for the item to show up in 1P mini.

jxpx777

Thanks for trying that @zupo and reporting that it didn't work for you… For now, my explanation of the behavior will have to suffice as I don't think this is something we will be changing. I'm sorry I don't have a better answer for your particular use case at the moment. :(

lsmith

"You can try setting your URL to just herokuapp.com and see if that helps"

That obviously does not help.

So you basically are saying you don't care about the use case of herokuapp.com/platform.sh etc?

matthew_ag

Hey @lsmith,

As I understand it, the herokuapp.com domain is only used so that subdomains like foo-pr-123.herokuapp.com can be used to refer to independent apps that run on the Heroku platform. These apps in most cases are completely independent from each other because the domain heroku.com domain is shared between all Heroku users and so if they do require a login, that login is unrelated to other apps in other Heroku sub-domains. For this reason, it's a good thing for the non developer user that 1Password automatically distinguishes each subdomain on herokuapp.com as unique because if they have signed up to two different apps hosted on the Herokuapp.com domain, the user won't have to choose between the unrelated Login items every time they need to log in.

If we remove herokuapp.com from our copy of the public suffix that 1Password includes then we break this nice feature for most of the users of herokuapp.com (as I understand it, this is a publicly used domain and the vast majority of users are not developers with this issue).

However I do understand that in your circumstances as a developer where you have a test environment that spins up apps that are dynamically named 1Password is making life difficult. While I'm not sure the following makes sense as I do not have any experience working with the Heroku environment, please let me suggest a workaround that you could try:

I found some Heroku documentation which describes how to specify Custom Domains for your apps. From what I can tell, using a custom domain would solve this issue and would involve two steps on your side:

1. Registering a new domain for testing or a test subdomain, e.g. test.yourdomain.com. This should only have to be done once.
2. Modifying your Heroku test app build deployment scripts so that the dynamic Heroku test app domain is made available on your test subdomain. So the new test app is available at foo-pr-123.test.yourdomain.com

Once this is completed, your test apps will always be available to your on your *.test.yourdomain.com and you will be able to create a Login item that you'll be able to use for all dynamically created subdomains.

I hope this helps - please let me know if this approach would make sense for your scenario. It certainly would be useful for others to hear if it was.

Thanks for writing in.

Best regards,
Matthew

ryanwilke

I just tried using the following wildcard in the website field and noticed this feature doesn't appear to be working yet: https://*.herokuapp.com/

Judging by the age of this ticket and the number of people that have been requesting this feature over the past few years, I suppose I probably shouldn't put my hopes high that you'll be addressing this issue anytime soon?

Let me know if there's something I'm missing, because I'd absolutely love this feature (as would many others I take it).

littlebobbytables

Hello @ryanwilke,

It is unlikely we will be adding wildcard support. We already support matching against any subdomains and in most cases this is sufficient. Heroku is an oddity because they explicitly got themselves added to Mozilla's list of public suffixes (see Jamie's post earlier) so that instead of treating everything under herokuapp.com as subdomains of the registered site herokuapp.com that x.herokuapp.com is treated as the registered name and that anything below x.herokuapp.com are subdomains. The entire point of this is so that x.herokuapp.com and y.herokuapp.com are considered completely different by anybody using the public suffix list. It makes the fact that by default they dynamically generate new names at that level odd to say the least.

The best option for handling this would be Matthew's suggestion of creating a custom domain so that all of the dynamically generated names appear as proper subdomains. If done 1Password will match a Login item for x.herokuapp.com to anything that looks likes zy.x.herokuapp.com.

ryanwilke

Thanks for the additional details. I'm a little confused still as to why it's so difficult to match the heroku subdomains, but that aside you did mention that 1Password already supports matching against some subdomains. How do those subdomains have to be formatted in order for the matching to work?

littlebobbytables

Hi @ryanwilke,

Sorry, I'll do my best to elaborate and hopefully that will help :smile:

Let's say you own a domain and you've registered the domain name mydomain for com. The Mozilla public suffix allows us to identify that com is the public suffix and we can then infer mydomain is the registered domain name. It's this list that ensures we don't mess up when looking at something like mydomain.co.uk where the actual registered domain name is at the third level and not the second.

Anything below the registered domain name is considered a subdomain. 1Password will first look for a match against the FQDN (Fully Qualified Domain Name) but if not we will then look at the registered domain name plus public suffix. So let's say you're visiting subdomain-A.mydomain.com. 1Password will first look to see if you have a Login item that has subdomain-A.mydomain.com stored in a website field and then to see if any Login items have just mydomain.com stored in a website field.

If 1Password find a match for the FQDN (Fully Qualified Domain Name) we will display it and hide the others behind the Show x more Items menu option. If there isn't a precise match we will display the partial matches. So you could have multiple subdomains and if you have a single Login item that matches against mydomain.com then it would be shown for all of them. It's this behaviour I'm referring to when I said we already support matching against subdomains.

The Mozilla public suffix list goes beyond listing just the known TLDs (Top-Level Domains) and SLDs (Second-Level Domains), they allow people to apply to be listed in such a way that anything using the list treats them just the same as a TLD. Heroku is one of these. They explicitly asked to be added to this list and the result is that instead of looking at herokuapp.com and seeing herokuapp as the registered domain name under the public suffix com they want people to treat the entire herokuapp.com as a public suffix. What this means is instead of the subdomain under herokuapp.com being viewed as a subdomain it is instead the equivalent to a registered domain name.

Given this entry, something like https://*.herokuapp.com/ would be essentially equivalent to https://*.com/ and you definitely wouldn't want 1Password allowing that.

If it's still unclear don't feel too disheartened. We had a call about this here at AgileBits and I kept asking "but why?" and in the end I still found it have my a headache. Matthew's discovery tames Heroku by ensuring that all of the dynamically generated subdomains are perceived as actual subdomains rather than being viewed as the equivalent to unique registered domain names.

I could kind of understand of the likes of WordPress did something like this so that each WordPress site was viewed as being a unique entity but given Heroku dynamically generate all these subdomains directly under herokuapp.com I don't get it. Maybe there's a great reason and if so it would help me rationalise it all :smile:

I hope this helps.

ryanwilke

Yep. That definitely helps me understand the technical aspects of it now. It sounds like you're not doing just a simple text matching of the string stored in the URL fields, you're actually taking it a step farther (I'm assuming for some kind of security reasons?). Simply matching a string with some simple regex wouldn't be very complicated for many people (especially developers) to understand, but if that's not how it's built then I guess I can understand why it's harder than it sounds. I think that's just how everyone would assume is how it's built.

jxpx777

And you wouldn't be wrong in assuming that. :) Time was, this was basically what we had. But, we found that as more and more people used 1Password in international locales this approach fell over and pretty hard. What's the most obvious thing? Take the last two components and that's the root domain! Except, you have things like co.uk and other ccTLDs. So, we went looking for a list of these TLDs and we found the Mozilla list I mentioned earlier and this was more comprehensive than anything we could have devised on our own.

The ability to proactively prevent filling of one Heroku app's credentials (to stick with that example) on another Heroku app page is indeed something that would fall under security, specifically our phishing protections. So, imagine you run an app through Heroku. Users can sign in there and post cute cat pictures or whatever. Imagine one of your users uses 1Password and gets tricked into visiting a phishing page also hosted on Heroku that is set up to look exactly like your cat pictures site.

If badapp.heroku.com looked just like cutecats.heroku.com and we didn't have this special rule in our domain matching, then by our other rules of Login matching, the cutecats Login would be shown when the user is viewing badapp (because herokuapp.com is in both Logins) and would think it was OK to fill their username and password there. In reality, the credentials have been delivered to badapp and now that person can sign in to cutecats, and, depending on their level of permission, make all kinds of trouble.

That's just one example, but the suffix rules we follow from Mozilla help us mitigate these kinds of risks in addition to helping us improve the usability of 1Password's Login matching.

--
Jamie Phelps
Code Wrangler @ AgileBits
Fort Worth, Texas

eberkund

Another +1 for wildcard support in the auto save exclusion list!

matthew_ag

Hey @eberkund,

Welcome to the forum and thanks for writing in. Wildcard support is being requested from a filling perspective in this thread rather than from a Autosave perspective - I'm curious to understand in what scenario you would need wildcard support in the Autosave exclusion list? If you add a website to the Autosave exclusion list, e.g. google.com, that will prevent Autosave prompts for all subdomains for Google. Would that help with what you're trying to do?

If not, perhaps you have an example you could share that would help me understand your request?

Looking forward to hearing back.

Best regards,
Matthew

eberkund

@matthew_ag Sure, I do a lot of web development and where I am testing with sample data that I don't want saved and it gets annoying when 1P prompts me everytime to save. It would be nice to be able to put *.dev in my exclude list.

Maybe it would have been better posted here: https://discussions.agilebits.com/discussion/29172/how-to-use-ip-ranges-from-excluded-domain-list actually I thought this thread was discussing the same thing.

jxpx777

I think the reason *.dev doesn't work is because it's one level beyond where 1Password looks for exclusions. We currently match the full hostname and the root domain name. We don't do any further matching beyond that, so *.dev would be left out. Right now, we don't have a great answer for your use case. You'll either need to get very friendly with the esc key or turn off autosave and use manual save when you actually want to save new Logins. We'll definitely keep this request in mind, though, as we make future improvements.

Thanks!

--
Jamie Phelps
Code Wrangler @ AgileBits
Fort Worth, Texas

matthewfallshaw

On 1Password's approach to wildcards:
There is a lot of customer dissatisfaction here. Not only is this thread three pages long, but there are other threads talking about this (or nearly this) on this forum going back years. You clearly have a large group of users who want a better solution to website matching, and you are not satisfying them.
Now, what you clearly (and very reasonably) don't want to do is complicate your user interface for less advanced users, but adding more advanced pattern matching doesn't seem to me like it will do that. Less advanced users simply won't add wildcards to URLs, and when more advanced users do add wildcards… what else could they possibly mean but "treat the '*' I just typed as a wildcard"? Adding wildcard support would not be a complicated feature to add, and it would satisfy most of those posting on this thread and the many other threads on this forum. Please reconsider adding wildcard support.

A different feature request (a useful feature to add with or without wildcard matching):
The feature - pay attention, when I invoke 1Password from a particular URL, to the passwords I choose when offered a list to choose from. Sort the list of options by frequency of my past choices rather than alphabetically.
Rationale for the feature - the 1Password extension often offers users a list of password matches. If the extension has no information but the "website" field recorded against each password then alphabetical sorting of matched password domains isn't a terrible choice. But, with or without wildcard matching, every time a user selects one match from that list, they just provided the extension with valuable information - they just said "whatever the domain match, when I'm on this particular URL I sometimes want this particular password". If you recorded, say, the last 10 precise URLs matched for each password, and you found that when offered {this list of passwords} for {this particular URL} (specific URL, not domain) they had chosen one particular password ten times from the last ten times they were offered the list and never any of the other passwords… then that's probably the one they're going to choose this time too.
And if, when offered this list of passwords for {this particular URL} you found that they'd chosen one particular password six times, a different password three times, and a different password once… they're much more likely to prefer a list of passwords sorted by the frequency of their past use over an alphabetically sorted list.
It sounds to me like this feature would not require any user configuration but would Just Work for most of those on this thread the the many other threads on this forum seeking wildcard matching.

(I'd love you to provide both of these features, but my use of 1Password would be much improved if you provided either one of them.)

AGAlumB

@matthewfallshaw: As I'm sure you're aware, both from this discussion, others you've referred to, and also your own experience as a more technical user, our first priority is to keep up with the major browsers, which are constantly evolving, so that any of this works at all in the first place; and a close second to that is our responsibility to continually improve 1Password's filling, which is by far the feature that 1Password users interact with most overall. Given these considerations, this isn't something we can work on right now. It's something we'll continue to consider for the future, but advanced features, even those we'd benefit from ourselves, which are not important to the vast majority of 1Password users, must needs defer to work that offers a greater good for many more people. Any user-facing feature, though perhaps superficially not "complicated", is just the tip of the iceberg; the development, testing, and support commitment inherent in any new feature is something we have to consider as well, not just the fact that there's a desire for it from a few passionate users.