OpenID Connect and WS-Fed OWIN Components: Design Principles, Object Model and Pipeline

After having promised (to you and to myself) to write more in depth about the new OWIN components for OpenId Connect and WS-Federation, I am finally carving out some time to sit down and jolt down my thoughts about it.

My goals for this post are to help you to understand what those new components are for, how they came to be and how they are structured. I am probably going to ramble a lot – I’ve been wanting to work on something like those libraries for the last 5 years or so, hence I have a backstory for practically every nook and cranny in fact, the tone of this post might make you think I am suggesting that the library is all my doing. That is not the intention, and above all that is not true, far from it. I brought my thoughts to the table, and the team (mostly the awesome duo Brent & Chris) brought theirs: the library is the combination of those, and to be fair they wrote all the code!

Goals

I already mentioned in the announcement post some of the drivers that made us decide to move past WIF’s programming model, hence I won’t repeat those here. I’ll only restate that WIF was really designed to provide a .NET implementation of the WS-Federation protocol, and that goal was evident pervasively in all its programming model. The tooling empowered developers not familiar with the protocol to be productive anyway, but only if they were OK with the default settings: anything more than that required specialized knowledge.
During my half decade spent touring the planet and explaining claims based identity to tens of thousands of people, I experienced firsthand that it is not the way most developers want to operate – and that shaped the ideal of what the new library should look like. The library should offer a smooth continuum between two different goals.

A library for web sign on

One goal of the library – probably the primary one – is to enable any developer to perform web sign on in his/her apps. To that end, no protocol specific knowledge should be required for successfully implementing the canonical scenario – apart from “I am using protocol X” and “those are the couple of values I need to initialize the software”. The protocol used is not a secret, but no knowledge beyond the aforementioned 2 points should be required.

To use a bit of a cliché, you can summarize this goal with “common scenarios should be easy”.

A highly extensible protocol library

If you read about ADAL’s design principles, you know that we explicitly avoided writing a protocol library –and that we didn’t add any extensibility points. That design is opportune for a client library whose main raison d’etre is requesting tokens, but isn’t going to fly in a library meant to protect resources on the service side.

For that reason: the OWIN components for claims based identity are also meant to help the developer to take advantage of specific features of the protocols it leverage, and to exercise control over aspects of the authentication and app lifecycle. Those have meaningful defaults in the canonical scenario, but those defaults might be changed to accommodate more specific scenario requirements.
In those cases, the developer should experience a graceful slope that progressively require more awareness of the protocol details and authentication pipeline which enable or touch the function he/she is after. Those more advanced uses might pivot around protocol knowledge (“I know OpenId Connect allows me to get a code together with the id_token, hence I expect to be able to configure the middleware to make that happen and expose the results to me in usable fashion”) or about application lifecycle knowledge (“I want to sign out, hence the library must have a gesture I can use for that without having to know in details how that happens – unless I want to customize how it is done”). We know that WIF’s richness has been one of the factors that helped bring claims based identity to the mainstream, and we certainly don’t want to lose it: but we believe it is possible to preserve (or even improve) such expressive power without extolling expensive complexity downpayments upfront.

Those two goals have been my North star at every step of the way.
For a generic developer this is a web sign on framework for ASP.NET, which is why so much energy has gone in having a coherent model (pipeline, way of specifying options) that spans all authentication protocols supported today (WS-Fed, OIDC) and any future one we’ll implement.
For the Dominicks and the Damians of the world, this is an OIDC and a WS-Fed library – which might have a feature set skewed toward the capabilities of Azure AD, but that ultimately will grow into a more complete protocol implementation.

Design Principles

There are many ways to go from A to B. To keep the trajectory as straight as possible I have been nagging everybody with the principles below. Knowing those principles might not be very actionable to you, but I think it might be useful to understand scope and intended use of the product.

The new OM should be native to ASP.NET: use its programming convention, ship within it, work well in combination with the existing components.
Simplicity. Tooling should be used only for alleviate menial tasks, never to hide complexity. If necessary, move the logic that is in the tooling today (e.g. consuming metadata) directly into the new library
- Corollary 1: always demand only the essential info to perform a given task and nothing more
- Corollary 2: avoid duplication. If a piece of data was already provided at some point, find a way of making it available where it is needed rather than asking the developer to provide it again in more places
Expressive power without compromising simplicity. Offer extensibility points that developers can use to control how the authentication flows, but implement them as opt-in features that can be used to refine the simpler scenarios without making them more complex when they are not needed
Don’t reinvent the wheel. A lot of the necessary capabilities (ClaimsPrincipal, token handling, OWIN conventions, etc) and practices are already available and in use. Don’t rewrite/override unless there’s a clear advantage in doing so
No backcompat for things that would carry forward complexity with little reward (e.g. IssuerNameRegistry) or reduce scope (e.g. WIF’s hard dependency on web.config)
Aggressive scoping. Focus on enabling RPs with ASP.NET.
WCF integration is out of scope. Creating issuers and STSes is out of scope.
Enable AD and implement the standards. The primary identity provider we are looking at is AD, both in its Azure and ADFS forms. We do want to implement the standards properly and interoperate, hence this influences just the order with which features will light up
Incorporate lessons learned about which extensibility points are most used in WIF and ensure that they are available in the new model too, in the form that make the most sense in the new model
Consistency. There is a high level structure that is common to every sign on protocol. The library should reflect that, so that implementations of different protocols share as much infrastructure as possible and developers don’t have to start from scratch every time they change protocols
Maintain protocol logic and session management logic separate. In WIF they were coupled more tightly than we would have liked, we don’t want to repeat that. Different protocol implementations should be able to leverage the same session infrastructure
Maintain pipeline management logic (e.g. OWIN) and protocol constructs (messages, tokens) in separate assemblies. A developer wanting to generate a ws-fed message should not be forced to bring in OWIN components to have access to the necessary logic
Use protocol construct names as described in the specs to name corresponding properties in the OM. In WIF we reinterpreted wreply and whr with “English” names like Reply and HomeRealm, but that caused confusion more than it helped.
Do no require changes in trust relationships. Developers should be able to update an app from WIF to the new model without having to go back to the identity provider and renegotiate trust relationships or make any protocol level changes.

If you consider that we were reimagining a piece of software in veeeery wide use, sticking to the above was quite the tall order! I might be biased, but I believe that the team truly delivered.nor that, it’s time for me to stop blabbering in abstract and get in the details of how the new components work.

Structure

When we set up to refactor the claims based identity programming model, ASP.NET already had its own model for delivering authentication capabilities. It was in the form of a collection of OWIN modules, called ‘middleware’, designed to sit in front of ASP.NET app in an OWIN pipeline and take care of authenticating requests before letting them through to the application’s code.

The most important of the ASP.NET middlewares is almost certainly CookieAuthentication, which takes care of both session tracking (by generating and validating session cookies) and basic authentication functions that were until recently addressed by the Forms authentication subsystem. We clearly didn’t care much about the forms authentication equivalent, given that we wanted to implement specific protocols – but it made perfect sense to leverage CookieAuthentication for taking care of sessions after the protocol dance took place. Now we are getting somewhere! We decided that we would create a separate middleware for each protocol we wanted to implement, and arrange things so that those new modules could take advantage of CookieAuthentication to maintain a session. Using the conventional extension methods which which the OWIN pipeline is traditionally initialized, we expected to have setup code of the following form:

app.UseCookieAuthentication(new CookieAuthenticationOptions());
app.UsePROTOCOLNAMEAuthentication(new PROTOCOLNAMEAuthenticationOptions(/*.STUFF.*/));

The way in which I understood this at first was as a modern equivalent of WIF’s WSFederationAuthenticationModule and SessionAuthenticationModule (FAM & SAM), however Chris eventually made me understand that although there is a functional correspondence, an important difference is that here the cookie middleware is the one in the driver’s seat – the session is the highest order bit.

As of today, we have middleware for WS-Federation and OpenId Connect. They live in the Microsoft.Owin.Security.WSFederation and Microsoft.Owin.Security.OpenIdConnect nugets, respectively.

Options

The above is used to weave in front of the application the authentication pipeline: but how do we feed to it the info it needs? What is the correspondent of the <identityModel/> settings in this new world?
You have seen above that the protocol middleware is initialized by an instance of a corresponding AuthenticationOptions class. OWIN defines a basic AuthenticationOptions, which is largely concerned with information used to integrate in the OWIN pipeline. We decided to derive from that a subclass for each authentication protocol, and put in there the protocol-specific information required for each middleware to drive the protocol dance it was meant to implement: the WS-Federation would get stuff like realm and wreply, the OpenId Connect one client_id and redirect_uri.
Those option classes are very comprehensive, however you don’t need to populate every single property in there; in fact, most of the times you don’t even need to know that there are properties besides the basic ones. To make a concrete example, here there’s how to init an OpenId Connect-protected app:

app.UseCookieAuthentication(new CookieAuthenticationOptions());
app.UseOpenIdConnectAuthentication(
    new OpenIdConnectAuthenticationOptions
    {
        Client_Id = "7397e47d-2745-48c1-80fd-fe7dd97fc376"
        MetadataAddress = "https://login.windows.net/developertenant.onmicrosoft.com/.well-known/openid-configuration"
     });

That’s right, those two properties are all you need. As you have read in the original announcement, the new middleware leverages the metadata document of the identity provider to fill in dynamically all the details that are necessary for driving the protocol dance and token validation. If you’ll keep a reference to that Options instance, you’d see that after initialization you’d see many other values filled besides those two. Just to give you a feeling for it, here there’s the complete list of properties:

We briefly considered identifying functional equivalences across protocols (wrealm for WS-Federation and client_id for OpenId Connect are both app identifiers, for instance), finding new names and put those in a common class, but per the aforementioned design principles we discarded the idea in favor of an OM that would not be confusing for the people who know the protocols.

However, we did find common, protocol-agnostic functionality that we could model consistently across different middlewares:

Token validation. The principles of token validation, and the information required to perform it, remains largely the same regardless of the protocol. Hence, we grouped all the relevant properties in one type – TokenValidationParameters – and we added a property of that type in each protocol options class
Request validation stages. As described here, today’s authentication protocols share more or less the same phases: redirect to the identity provider, parsing of requests and retrieval of the token, token validation, session creation and so on. WIF offered to developers the opportunity of injecting custom logic in each of those stages, via event handlers. Here we offer the same feature by exposing a set of delegates, which we call notifications, that you can latch to to the same effect. Each protocol has its own corresponding Notifications type, and each Options has a Notification property of that type. The notifications for WS-Federation and OpenId Connect are almost the same.

Let’s consider each of those constructs separately.

TokenValidationParameters

We already had a foray in the OWIN world when we first shipped the Web API middleware. The API middleware, the one with the world’s longest extension method name (UseWindowsAzureActiveDirectoryBearerAuthentication) is essentially a thin wrapper of the JWT token handler – it automatically retrieves the token from the request and it (also automatically) read Azure AD’s metadata to download signature verification keys and know how to validate tokens, but the heavy lifting is done by the handler itself.

You might recall that at the time we made it an explicit goal to make the JWT handler usable outside of the WIF’s pipeline. To break the dependency from web.config elements, we created a new container class for all the validation info and wired the handler to use that for sourcing the info it needed. That class was the first incarnation of TokenValidationParameters (TVP for convenience).

For the new middleware we wanted to do the same, hence we reused TVP for the same function. Also, we took the opportunity to improve its design, based on the lessons learned in the months of use in web API. Here there’s how it looks like at the moment:

Most of those values will be obscure for you, and that’s perfectly normal: you very rarely need to operate a this level, it is warranted mostly in cases in which you need to validate custom tokens from sources that do not support metadata hence you want to feed signing keys manually.

The exception to that is represented by simple properties such as SaveSignInToken, a switch corresponding to WIF’s old saveBootstrapContext, and by the Func typed members AudienceValidator, IssuerSigningKeyRetriever and IssuerValidator.
You know from this post that validating a token boils down to verifying signature, issuer and audience. Most of the time those values are fixed and are read at init time from metadata and direct config, however there are important scenarios where that is not the case. For example, if your app is multitenant the tokens that should be considered valid have Issuer values coming from a dynamic list – which means that your validation logic must be customized to verify against a dynamic list rather than a single, fixed value. The IssuerValidator notification allows you to do just that very easily. For example:

app.UseOpenIdConnectAuthentication(
    new OpenIdConnectAuthenticationOptions
    {
        Client_Id = ClientId,
        MetadataAddress = OIDCMetadata,
        TokenValidationParameters = new System.IdentityModel.Tokens.TokenValidationParameters
        {                        
            IssuerValidator = (issuer, token) =>
                {
                    return DoesIssuerBelongToMyCustomersList(issuer);
                },
         }
//...

Super easy. Compare that to having to produce a custom IssuerNameRegistry as you had to do in the old WIF model

Note that instead of customizing you can also opt for altogether shutting down certain token validation functions. You would do that when you want to take care of that function later in the pipeline. For example: if your app accepts calls from specific users, rather than from specific issuer, you’ll want to turn off issuer validation and add validation logic at a later stage of the pipeline, where the token has been already turned in a ClaimsIdentity and you can perform your user lookup through its claims.

Notifications

As it was the case for WIF’s HttpModules events, Notifications offer you the ability of injecting your own logic at key stages of the request validation pipeline. They are very handy for a number of advanced tasks, and they are very easy to use. Here there’s the list for OpenId Connect (remember that we are still in preview and things might change before GA!).

Here there’s the list of the ones we currently have in the preview for OpenId Connect – the WS-Federation ones are practically the same apart from the AccessCodeReceived which is OpenId specific. Note that some of those notifications aren’t wired up for the time being (we are still not 100% sure about SignedIn and SignedOut).

Here there’s a simple diagram of how the notifications unfold as the request processing progresses. The left to right arrow represents an incoming request, the right to left arrow represents any redirect headed to the identity provider. The order might reflect the current nightly builds hence not be 100% aligned with the preview.

Here there’s a quick comment for each notification.

RedirectToIdentityProvider

This is the exact equivalent of the near homophone in WIF.

This notification fires every time you are redirected toward the identity provider – today that means either during a sign in or a sign out flow.

You can use this notification for dynamically modifying the message (available in the ProtocolMessage property) to the identity provider with information that was not available at initialization time, like indications on the home realm of the caller or a dynamic signout URL based on where the app gets deployed. For example:

 app.UseOpenIdConnectAuthentication(
  new OpenIdConnectAuthenticationOptions
  {
     Client_Id = ClientId,
     Authority = Authority,                    
     Notifications = new OpenIdConnectAuthenticationNotifications()
     {
        RedirectToIdentityProvider = (context) =>
       {
           string appBaseUrl = context.Request.Scheme + "://" + context.Request.Host + context.Request.PathBase;                
           context.ProtocolMessage.Post_Logout_Redirect_Uri = appBaseUrl;
           return Task.FromResult(0);
         },
//...

The WS-Federation middleware works in exactly the same way, but of course the ProtocolMessage property there reflects the different parameters used in that flow.

MessageReceived

This notification fires as soon as a protocol middleware recognizes that the incoming message is a protocol message of the type it is competent for and for the resources/AuthenticationType it is configured for.

The message is available in its totality. You’ll want to handle this notification when the message contains something that you already know our middleware won’t handle the way you want it handled. For example, Dominick is using this to handle response_types we don’t yet support; or you might have custom extensions to the protocol or any extra app specific context you might want to process.

SecurityTokenReceived

This notification fires just after a token has been identified and extracted from the incoming request. If you have token specific logic you want to run, this is the place.

From the TVP: AudienceValidator, IssuerValidator

Those are not notifications from the protocol notifications properties: they are the delegates from the TVP, as described earlier. I am listing them here because if you see this list as a sequence of event, this is the point in which they would fire.

SecurityTokenValidated

Once the token has been parsed and validated, its content is used to construct an AuthenticationTicket with its accompanying ClaimsIdentity; then this notification fires.

You’ll want to use this notification for any logic that requires you to have access to the caller’s identity and that you want to execute in the pipeline, before your own app code is reached.

Typical uses of this notification include

the already mentioned user check against a db of valid users. Example here
any claims augmentation logic – you can simply add to the ClaimsIdentity in the context

We did not include direct support for ClaimsAuthenticationManager, given that there’s some impedance mismatch (ClaimsPrincipal vs ClaimsIdentity) and that doing “claims math” seems easy enough directly in this notification. If you have an opinion please let us know!

AccessCodeReceived

This event is only available in OpenId Connect: it fires when the authorization server sent over an authorization code. As the code is single use, if you want to redeem that code for a token your only opportunity is to add the necessary logic in this notification. Example here.

Note: this notification will likely change name to AuthorizationCodeReceived.

AuthenticationFailed

If something goes wrong during the validation, this notification will be fired. Here you have the opportunity to do some remediation and/or error management – and avoid presenting a YSOD to the user. For example:

AuthenticationFailed = (context) =>
{
   context.Redirect("/Home/Error");
   return Task.FromResult(0);
}

Dependencies

As mentioned, we layered things so that protocol constructs are isolated from OWIN middleware, and vice versa. The protocol middleware depend on two other nugets:

System.IdentityModel.Tokens.Jwt

This is the next version of the good old JWT handler, modified to accommodate the new capabilities we require in the new OM.

Microsoft.IdentityModel.Protocol.Extensions

This is a new assembly which houses all the low level and protocol specific constructs we need. It includes message types for all protocols, a wrapper for using the old token handlers (like SAML) without web.config dependencies, and other goodies.

Wrap

Well, that was a long post! I hope this will help the ones among you that want to go beyond the basics, and I hope it will provide a reading key for longtime WIF users who are trying to grok the new model.

I cannot stress enough that this work was a team effort, with Brent and Chris being the superstars that made the magic happen. As we are getting closer to lock this for release, we are eager to get your feedback to ensure that this works great for you!