May 2014 cumulative update (CU) changes to SharePoint Server 2013 hybrid
Applies to: SharePoint Server 2013
Topic Last Modified: 2014-05-26
Summary: The May 2014 cumulative update (CU) for SharePoint Server 2013 includes a code change to the SharePoint authentication service that gives farm administrators greater control over OAuth request validation behavior. If you need these updates follow the instructions in the two KB Articles below:
The big news is that, in an inbound or two-way SharePoint hybrid, the code change lets hybrid users outside the on-premises intranet securely access on-premises content. When hybrid users with the permission to see the information click an on-premises result in combined SharePoint Online (SPO) search results, they will be able to open the document without needing an active VPN or DirectAccess connection to the intranet.
Imagine that your company has SharePoint hybrid users who work remotely; for example, from hotel rooms or airports during visits to customer sites. If these users click on-premises links in their search results, they must be connected to the corporate network using one of the following technologies to open a document:
Otherwise, these requests would return a 403 Forbidden error message. In fact, clicking an on-premises search result would return a 403 error to a user on any network outside the reverse proxy.The challenge was to simplify this user scenario so that it acted more like an intranet/on-premises users’ experience. This is what drove the code change.
SharePoint hybrid architectures are based on a server-to-server (S2S) trust relationship between SharePoint 2013 and Office 365. SharePoint 2013 uses OAuth 2.0 to establish this trust.
OAuth works by passing a bearer access token that contains a user claim to the resource server. The resource server authorizes the requested transaction on behalf of the user. OAuth must be able to validate some key information to construct a token that the client and resource can use to communicate. Some technical details are described here for context, but we’ve omitted details that aren’t relevant to the issue.
When SharePoint Server gets a search query request from SPO, it returns an HTTP 401 challenge with a bearer token. SPO sends the token back with the URL of the SharePoint farm to which it is sending the request, among other values. The SharePoint authentication service checks whether the original request URL in the bearer token matches the public URL of the web applications in the farm. If there is no web application that has a matching public URL, the authentication service denies the request and sends a 403 error response to the client.
This problem has three elements:
The request URL (which is the value of the audience claim in the OAuth bearer token) must exactly match the public URL of the destination web application. This is an OAuth requirement.
Traffic from SPO must be relayed to the on-premises SharePoint farm by a reverse proxy, and it must be configured to pre-authenticate all inbound traffic with client certificate authentication. This is required in inbound and two-way SharePoint hybrids.
The URLs of SharePoint pages and content that a remote user sees always begin with the public URL of the web application that contains the site collection. This is required for public DNS routing.
In an inbound hybrid search topology, SPO queries the on-premises SharePoint farm using a public URL (for example, https://spexternal.adventureworks.com). This URL resolves to an endpoint on a reverse proxy that’s configured to first pre-authenticate requests from SPO with a client authentication certificate, and then relay the request to the SharePoint farm. Client certificate authentication between SPO and the reverse proxy is a required security method for all inbound hybrid topologies.
After the query reaches the on-premises SharePoint farm and is processed, search results are sent back to SPO. SharePoint hybrids render content to remote users by using the web application’s public URL. This includes rendering search results URLs, for example https://spexternal.adventureworks.com/<path>. But remote users wouldn’t be able to access content at this URL because, even though this is a publicly resolvable URL, it routes all requests to an intranet site inside an organization. Also, they wouldn’t have the client authentication certificate that would let them pass through the reverse proxy.
If you’re not familiar with them, alternate access mappings or AAMs are used in SharePoint to define what URLs have access to a SharePoint site. They are also used to return a proper URL for a proper access zone (such as Internet, Extranet, or Intranet). Every site has at least a default AAM. This can be a URL that is registered in internal DNS, such as https://sharePoint, or a URL that can be registered in both internal and external DNS, like https://spexternal.adventureworks.com. Depending on how you created the SharePoint site, you may have a publically resolvable URL both inside and outside of your reverse proxy.
If the internal and external URLs for your SharePoint site aren’t the same, you can associate up to five public URLs, URLs that are resolvable by public DNS, with a single SharePoint web application through AAMs. This means if the public URL you purchased is different from the internal URL of the site, the public URL can be linked to the web application by adding it in a zone (for example, the Internet zone).
Also, if you are routing Internet traffic to an internal SharePoint site but you have to terminate the public URL on the reverse proxy, you can ‘Add Internal URLs’ to a zone. This is an alternate URL mapping recognized by SharePoint that can return the public URL back to external users. In this case, you would be using the Internal URL to bridge the distance from the reverse proxy to SharePoint Server, which is where the term bridging URL comes from. You can see the Default zone URL, and two Extranet zone URLs below. The second Extranet entry is for a bridging URL. Internally, the site can be accessed with ‘http://sharepoint’, but when SharePoint returns any dynamically generated URLs to users browsing http://sharepoint, they contain the public URL resolvable in public DNS. This means extranet users at the other side of the bridge get a URL that is properly resolvable for their access zone.
This is what the problem looks like for a user coming from outside of the domain.
An enterprise user, working remotely, authenticates to the company’s SPO search portal (https://adventureworks.sharepoint.com/search) and enters a search term.
SPO queries the on-premises SharePoint farm using the external URL (https://spexternal.adventureworks.com) that resolves to the reverse proxy endpoint.
The request is pre-authenticated using a shared client authentication certificate and the reverse proxy relays the request to the SharePoint farm using the internal URL https://sharepoint.
The SharePoint authentication service compares the original request URL in the request’s bearer token with the public URL of the web application. The values match, and SharePoint validates the request.
SharePoint processes the search request, security trims the results based on the permissions of the user account making the request, and returns the results to SPO.
Search results from both SPO and on-premises SharePoint Server are shown on the SPO search results page.
But when the remote user logs in and clicks an on-premises document link in the search results (https://spexternal.adventureworks.com/documents/document.docx), the request to open this document from its document library must also use the public URL that resolves to the reverse proxy endpoint (just as seen in the original SPO query).
The reverse proxy responds by requesting the client certificate from the user’s computer. Since this certificate is not present, the reverse proxy cannot pre-authenticate the request, and returns a 403: Forbidden error to the client computer. A process that looks like this:
Reverse proxy endpoint: https://spexternal.adventureworks.com
Hybrid site bridging URL: https://sharepoint
Hybrid site public URL: https://spexternal.adventureworks.com
May CU lets SharePoint farm administrators choose a solution to this dilemma. Either use the web application’s public URL, or a SharePoint alternate access mapping (AAM), to validate OAuth requests. After installation of May CU, farm administrators can switch between these two options:
Public URL matching: As it does natively, the SharePoint authentication service compares the original request URL with the public URL of the primary web application. These URLs must match for the request to succeed.
This option is ideal for host named site collection-based SharePoint sites, or path-based sites with the same default URL as the public URL.
AAM matching: Instead of using the original request URL for comparison (which may be a bridging URL used by the reverse proxy to relay the request) the authentication service compares this URL with the AAMs configured for the web application, and allows the request if a match is found. This feature can be enabled from the SharePoint Management Shell.
This option is helpful if the internal URL of a site does not match the public URL, such as in path-based sites where the default URL isn’t the same as the public URL or in any situation where the internal URL must not be public knowledge.
By adding a second reverse proxy endpoint, different Internet-routable URLs can be used to process requests from SPO, versus requests from users accessing on-premises SharePoint content.
Configure one reverse proxy endpoint to listen for SPO queries. These can be routed to a URL that does not match the public URL of the web application.
A second reverse proxy endpoint can be set up to listen for user requests for content. This endpoint can use the web application’s public URL, and can pre-authenticate user requests for on-premises SharePoint content by using Active Directory Federation Services (ADFS), Forms Based Authentication (FBA), or any other authentication methods that are available.
Here’s how the process works after May’s cumulative update.
An enterprise user, working remotely, authenticates to the organization’s SPO search portal (https://adventureworks.sharepoint.com/search) and enters a search term.
Three related things happen here: SPO queries itself, but also the on-premises SharePoint farm using the external URL (https://spo-query.adventureworks.com), which resolves to reverse proxy endpoint A. Also, this request is pre-authenticated using a shared client authentication certificate.
The reverse proxy relays the request to the SharePoint farm using the internal URL https://sharepoint.
The SharePoint authentication service compares the original request URL in the request’s bearer token with the list of internal URLs (AAMs) configured for the web application. The internal URL https://sharepoint is confirmed as an AAM in the web application, and SharePoint authenticates the request.
SharePoint parses the search index and does security trimming based on the user context.
Results are returned to SPO and, because the results URLs must be publicly routable, the URLs begin with the public URL of the web application (https://spexternal.adventureworks.com).
In the SPO search portal page, search results from both SPO and on-premises SharePoint are displayed.Now the user clicks on a search result. The document for that result is located in an on-premises SharePoint document library (https://spexternal.adventureworks.com/documents/document.docx). This new request goes to reverse proxy endpoint B.
The reverse proxy pre-authenticates the request using AD FS.
The reverse proxy relays the request to the SharePoint farm, again using the internal URL, https://sharepoint.
SharePoint matches the request URL against AAMs in the web application, and authenticates the request.
SharePoint returns the content to the user.
Reverse proxy endpoint A is for SPO queries: https://spo-query.adventureworks.com
Reverse proxy endpoint B is for user requests: https://spexternal.adventureworks.com
Hybrid site bridging URL AAM: https://sharepoint
Hybrid site public URL AAM: https://spexternal.adventureworks.com
To toggle AAM matching, you can run the PowerShell commands outlined here. These commands set the value of the property UseIncomingUriToValidateAudience to True. The default setting, which uses native public URL matching, is False.
To set AAM Matching for the entire farm, use this code:
$config = Get-SPSecurityTokenServiceConfig $config.UseIncomingUriToValidateAudience = $true $config.Update()
To set AAM Matching for a specific web application, use this code:
$webApp = Get-SPWebApplication <web application URL or ID> $webApp.UseIncomingUriToValidateAudience = $true $webApp.Update()
The web application setting (if it is configured) overrides the farm setting.