Mitigating Facebook’s “x-fb-http-engine: Liger” site hammering using Apache or nginx

Facebook is using a sort of web page prerendering mechanism in it’s in-app browser for mobile users [see disclosure here]. It is used by Facebook’s in-app browser to speed up page rendering for web link content shared on Facebook. But it can cause serious headaches, and this post should explain how to mitigate it with Apache or nginx.

Web page prerendering is not uncommon way of reducing frontend load times. Most today’s browsers support it, and some of them use it regularly (like Chrome – did you notice how first results on Google sometimes render instantly?!).  Some browsers prefer “preview” instead of “prerender” [like Safari], so other browsers [like Chrome] had to accept both instructions for the same action.

We all know that Magento CPU cycles come at a price. You should test on your own whether this FB in-app feature has implications on your web server performance and expenditure.

Facebook’s “Liger preview” requests can be recognized in your server logs by:

  • HTTP referer: “http://m.facebook.com
  • User-Agent has: FB_IAB/FB4A;FBAV (Facebook in-app browser for Android)
  • x-fb-http-engine: Liger – this is the identifier of the engine, BUT, FB in-app does not send it every time! It is being sent only when X-Purpose is sent:
  • X-Purpose: preview – page prerender/prefetcher HTTP header
  • regular images are not loaded – FB in-app requests only for HTML page

Every owner of Facebook page(s) that regularly shares links and with more than 10K fans should test their website for this particular page requests, as it might severely impact web site performance. Please, check this article on how to precisely log and test how severe your web site is affected.

I’m affected, what now?!

Since these requests are marked with two significant HTTP headers, we’ll use them to block this type of requests. We don’t want to block ALL prerender/preview requests, only those that come from Facebook’s in-app browser engine signed as “Liger“.

In Apache, you can use [tested, verified]:

RewriteCond %{HTTP:x-fb-http-engine} Liger
RewriteCond %{HTTP:X-Purpose} preview
RewriteRule .* - [F,L]

 

In nginx, you can use [not tested, but should work; please verify]:

This rule checks for presence of one HTTP header [faster]:

if ($http_x_fb_http_engine = "Liger") {
return 403 "Access denied due to Facebook's 'Liger preview' site hammering";
}

This one covers requests when both headers are present [since nginx does not support logical AND, we have to use a trick] :

if ($http_x_fb_http_engine = "Liger") {
set $lp L;
}
if ($http_x_purpose = "preview") {
set $lp "${lp}P";
}
if ($lp = "LP") {
return 403 "Access denied due to Facebook's 'Liger preview' site hammering";
}

And that’s it!

Don’t forget, prerendering can be quite cool and useful, so not all prerender/preview requests should be blocked – only those that originate from “Liger” engine. Prerendering advantages will be covered in the next article so – stay tuned!

Drazen Karacic-Soljic

- eCommerce Consultant

Cacan is an eCommerce Consultant at Inchoo, where he's monitoring key performance indicators of client's online store and suggesting improvements.

Read more posts by Drazen / Visit Drazen's profile

9 comments

  1. Thanks so much for this! I’ve been beating my head against the wall for weeks! Question, will blocking the LIGER preview cause FB to limit a post’s views or reach?

  2. I have found another engine that is causing invalid visits just like LIGER, in this case it seems the visits from ios mainly and the Ipad agent, it is about
    X-FB-HTTP-Engine: Tigon+iOS

    And my apache configuration in .htaccess is:

    RewriteCond %{HTTP:x-fb-http-engine} Liger
    RewriteRule .* – [F,L]
    RewriteCond %{HTTP:X-FB-HTTP-Engine} Tigon
    RewriteRule .* – [F,L]
    RewriteCond %{HTTP:X-FB-HTTP-Engine} iOS
    RewriteRule .* – [F,L]

    I think that way I mitigo much more the effects in the server … it would be interesting to share this postulate and to see if this motor is also affected to you and if at the same time it manages to mitigate more the effects

    regards

    1. Alfonso, are you checking RewriteCond %{HTTP:X-Purpose} preview for each of those 3 engines?

  3. Hi Drazen,
    the approach suggest to NEGATE the access to the request with those headers? Does this result in a user being unable to visit your site through FB app? or whenever FB in-app browser detect that someone reply with 403 try to load the page entirely from your webserver?

    I stumbled upon this article searching why I have much more nginx requests than the ones i see in google analytics, so i’d like to have all my traffic correctly shown on GA… is your approach viable to do this?

    Thanks in advance!
    C

    1. Hi Carmelo,

      thank you for your question. You are right – this blocks THIS type of requests – read more: http://inchoo.net/dev-talk/magento-website-hammering-facebook-liger/

      User will be able to visit site/page, the only drawback is that page will not be prerendered [preloaded], and user would have to wait for page to load.

      Yes, it’s viable, tested and works… if you have time, please, read original article and post your “liger rate”…

      Thank you…

    2. a little tip:
      instead of replying with 443 status, it is possible to use 444 that does not tackle at all the server 🙂

      Each cent of cpu is worth!

      Cheers and thanks for the great article!
      C

  4. Thanks for your post. This and your preview post helped us understand the increased server traffic from Facebook Android app users.

Leave a Reply to Carmelo Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <blockquote cite=""> <code> <del datetime=""> <em> <s> <strike> <strong>. You may use following syntax for source code: <pre><code>$current = "Inchoo";</code></pre>.