In the previous post Use Prerender to improve AngularJS SEO, I have explained different solutions at 3 different levels to implement Prerender.

In this post, I will explain how to implement a ASP.NET Core Middleware as a application level middleware to implement prerender. 


Application Level Middleware Architecture

At first, let's review what's the appliaction level middleware solution architecture.


ASP.NET Core Middleware - PrerenderMiddleware

In ASP.NET Core, we can create a Middleware, which has the similar functionality as HttpModule in ASP.NET, but in ASP.NET Core, there is no interface or base class we can use to declare a Middleware.

  • Create PrerenderMiddleware class

The default convention is that, we need to:

  1. The Middleware class needs to have a constructure which has RequestDelegate parameter as for next delegate.
  2. The Middleware class needs to have an async Invoke method with parameter HttpContext

So, the class is as below. I have added PrerenderConfiguration for getting configuration.

#region Ctor
public PrerenderMiddleware(RequestDelegate next, PrerenderConfiguration configuration)
    _next = next;
    Configuration = configuration;

#region Properties
public PrerenderConfiguration Configuration { get; private set; }

#region Invoke
public async Task Invoke(HttpContext httpContext)
    await Prerender(httpContext);
  • Then, we need to implement Prerender(httpContext) logic

            If you know my implementation for PrerenderHttpModule in ASP.NET, I used HttpWebRequest & HttpWebResponse.

           But for PrerenderMiddleware here, I use HttpClient, as with the HttpWebRequest in ASP.NET Core (at 2/11/2017), there is no way to setup AllowAutoRedirect and other http headers.

private async Task Prerender(HttpContext httpContext)
    var request = httpContext.Request;
    var response = httpContext.Response;
    var requestFeature = httpContext.Features.Get<IHttpRequestFeature>();
    if (IsValidForPrerenderPage(request, requestFeature))
        // generate URL
        var requestUrl = request.GetDisplayUrl();
        // if traffic is forwarded from https://, we convert http:// to https://.
        if (string.Equals(request.Headers[Constants.HttpHeader_XForwardedProto], Constants.HttpsProtocol, StringComparison.OrdinalIgnoreCase)
         && requestUrl.StartsWith(Constants.HttpProtocol, StringComparison.OrdinalIgnoreCase))
            requestUrl = Constants.HttpsProtocol + requestUrl.Substring(Constants.HttpProtocol.Length);
        var prerenderUrl = $"{Configuration.ServiceUrl.Trim('/')}/{requestUrl}";

        // use HttpClient instead of HttpWebRequest, as HttpClient has AllowAutoRedirect option.
        var httpClientHandler = new HttpClientHandler() { AllowAutoRedirect = true };
        // Proxy Information
        if (!string.IsNullOrEmpty(Configuration.ProxyUrl) && Configuration.ProxyPort > 0)
            httpClientHandler.Proxy = new WebProxy(Configuration.ProxyUrl, Configuration.ProxyPort);

        using (var httpClient = new HttpClient(httpClientHandler))
            httpClient.Timeout = TimeSpan.FromSeconds(60);
            httpClient.DefaultRequestHeaders.CacheControl = new System.Net.Http.Headers.CacheControlHeaderValue() { NoCache = true };
            httpClient.DefaultRequestHeaders.TryAddWithoutValidation(Constants.HttpHeader_ContentType, "text/html");
            httpClient.DefaultRequestHeaders.TryAddWithoutValidation(Constants.HttpHeader_UserAgent, request.Headers[Constants.HttpHeader_UserAgent].ToString());

            if (!string.IsNullOrEmpty(Configuration.Token))
                httpClient.DefaultRequestHeaders.TryAddWithoutValidation(Constants.HttpHeader_XPrerenderToken, Configuration.Token);

            using (var webMessage = await httpClient.GetAsync(prerenderUrl))
                var text = default(string);
                    response.StatusCode = (int)webMessage.StatusCode;
                    foreach (var keyValue in webMessage.Headers)
                        response.Headers[keyValue.Key] = new StringValues(keyValue.Value.ToArray());

                    using (var stream = await webMessage.Content.ReadAsStreamAsync())
                    using (var reader = new StreamReader(stream))
                        text = reader.ReadToEnd();
                catch (Exception e)
                    text = e.Message;
                await response.WriteAsync(text);
        await _next.Invoke(httpContext);
  • At last, let's take  a look at IsValidForPrerenderPage(HttpRequest request, IHttpRequestFeature requestFeature), This method is the same as PrerenderHttpModule class in ASP.NET.
private bool IsValidForPrerenderPage(HttpRequest request, IHttpRequestFeature requestFeature)
    var userAgent = request.Headers[Constants.HttpHeader_UserAgent];
    var rawUrl = requestFeature.RawTarget;
    var relativeUrl = request.Path.ToString();
    // check if follows google search engine suggestion
    if (request.Query.Keys.Any(a => a.Equals(Constants.EscapedFragment, StringComparison.OrdinalIgnoreCase)))
        return true;

    // check if has user agent
    if (string.IsNullOrEmpty(userAgent))
        return false;

    // check if it's crawler user agent.
    var crawlerUserAgentPattern = Configuration.CrawlerUserAgentPattern ?? Constants.CrawlerUserAgentPattern;
    if (string.IsNullOrEmpty(crawlerUserAgentPattern)
     || !Regex.IsMatch(userAgent, crawlerUserAgentPattern, RegexOptions.IgnorePatternWhitespace))
        return false;

    // check if the extenion matchs default extension
    if (Regex.IsMatch(relativeUrl, DefaultIgnoredExtensions, RegexOptions.IgnorePatternWhitespace))
        return false;

    if (!string.IsNullOrEmpty(Configuration.AdditionalExtensionPattern) && Regex.IsMatch(relativeUrl, Configuration.AdditionalExtensionPattern, RegexOptions.IgnorePatternWhitespace))
        return false;

    if (!string.IsNullOrEmpty(Configuration.BlackListPattern)
      && Regex.IsMatch(rawUrl, Configuration.BlackListPattern, RegexOptions.IgnorePatternWhitespace))
        return false;

    if (!string.IsNullOrEmpty(Configuration.WhiteListPattern)
      && Regex.IsMatch(rawUrl, Configuration.WhiteListPattern, RegexOptions.IgnorePatternWhitespace))
        return true;

    return false;



Use PrerenderMiddleware in ASP.NET Core Project

In order to use PrerenderMiddleware in ASP.NET Core project easily, I have created some extension method, so that we can easily setup it in Startup.cs

  • AddPrerenderConfig()

            AddPrerenderConfig is used to add PrerenderConfiguration.json to IApplicationBuilder.

        /// <summary>
        /// Add PrerenderConfiguration.json to configuration.
        /// Or you can put the configuration in appsettings.json file either.
        /// </summary>
        /// <param name="builder"></param>
        /// <param name="jsonFileName"></param>
        /// <returns></returns>
        public static IConfigurationBuilder AddPrerenderConfig(this IConfigurationBuilder builder, string jsonFileName = "PrerenderConfiguration.json")
         => builder.AddJsonFile(jsonFileName, false, true);
  • ConfigureSection()

             ConfigureSection is used to configure options into servicecollection, so that we can easily get it from servicecollection in the future.

        /// <summary>
        /// Configure Section into Service Collections
        /// </summary>
        /// <typeparam name="TOptions"></typeparam>
        /// <param name="serviceCollection"></param>
        /// <param name="configuration"></param>
        /// <param name="singletonOptions"></param>
        public static void ConfigureSection<TOptions>(this IServiceCollection serviceCollection, IConfiguration configuration, bool singletonOptions = true)
            where TOptions : class, new()

            if (singletonOptions)
                serviceCollection.AddSingleton<TOptions>(a => a.GetService<IOptions<TOptions>>().Value);
  • UsePrerender()

            UsePrerender is used to register PrerenderMiddleware

        #region UsePrerender
        /// <summary>
        /// Use Prerender Middleware to prerender JavaScript logic before turn back.
        /// </summary>
        /// <param name="app"></param>
        /// <param name="configuration">Prerender Configuration, if this parameter is NULL, will get the PrerenderConfiguration from ServiceCollection</param>
        /// <returns></returns>
        public static IApplicationBuilder UsePrerender(this IApplicationBuilder app, PrerenderConfiguration configuration = null)
            => app.UseMiddleware<PrerenderMiddleware>(configuration ?? app.ApplicationServices.GetService<IOptions<PrerenderConfiguration>>().Value);
         // => app.Use(next => new PrerenderMiddleware(next, configuration).Invoke);
         // => app.Use(next => context => new PrerenderMiddleware(next, configuration).Invoke(context));  // either way.        
  • With above extension methods, we can easily setup PrerenderMiddleware in Startup.cs


    • Step 1
public Startup(IHostingEnvironment env)
    var builder = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
        .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
        // Prerender Step 1: Add Prerender configuration Json file.
    Configuration = builder.Build();
  • Step 2
public void ConfigureServices(IServiceCollection services)
    // Add framework services.

    // Prerender Step 2: Add Options.
  • Step 3
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)

    // Prerender Step 3: UsePrerender, before others.

    if (env.IsDevelopment())




I have added PrerenderConfiguration.json file into ASP.NET Core project, then I can configure for prerender service.

The format of this json file is:

  "PrerenderConfiguration": {
    "ServiceUrl": "",
    "Token": null,
    "CrawlerUserAgentPattern": null,
    "WhiteListPattern": null,
    "BlackListPattern": "lib|css|js",
    "AdditionalExtensionPattern": null,
    "ProxyUrl": null,
    "ProxyPort": 80

You can go to my github wiki page to get more details about each option: Configuration & Check Priority  


Nuget Package

I have created a nuget package, which is very convenient if you don't want to dive deep into the source code. 

  • Install Nuget Package in your project.

           Visual Studio -> Tools -> Nuget Package Manager -> Package Manager Console.

Install-Package DotNetCoreOpen.PrerenderMiddleware

           If you want to take a look more detail about this package, you can go

  • Use PrerenderMiddleware and configure PrerenderConfiguration.json for prerender service.

            I have fully documented how to do this in my github wiki page, you can go there take a look.

  1. Prerender Middleware for ASP.NET Core

  2. Configuration & Check Priority            

  • Done, try it out.


Github Project

I also have created a github project to host all source code includes sample code for testing:, in this project, it includes ASP.NET HttpModule, ASP.NET Core Middleware, IIS Configuration 3 different solution. 

For ASP.NET Core Middleware, you can go to 


Prerender Related

  1. Use Prerender to improve AngularJS SEO
  2. Setup Prerender Service for JavaScript SEO
  3. Prerender Implementation Best Practice
  4. Prerender Application Level Middleware - ASP.NET HttpModule
  5. Prerender Application Level Middleware - ASP.NET Core Middleware


posted on 2017-02-13 16:47  博弈无涯  阅读(242)  评论(0编辑  收藏  举报