ted serbinski – entrepreneur & web architect
  • thoughts
  • about
  • contact



Popular content

  • Working 4 days a week
  • Yale School of Drama
  • How often do you change your oil? Maybe too often...
  • What happened to Menu bar module? Well, it got renamed...
  • Location data with Drupal
  • Drupal admin menu bar released
  • Our apt is haunted, update!
  • New OLF module, released from SF
  • Call Me "Driver S"
  • Preventing Drupal from Handling 404s for Performance
more

Recent comments

  • really interesting and well
    5 days 14 hours ago
  • cloning the
    3 weeks 5 days ago
  • vacabindex module (drupal
    4 weeks 1 day ago
  • I seem not to know where to
    4 weeks 2 days ago
  • Hi guys, Here a module
    5 weeks 6 days ago
more

Getting Drupal to Play Nice with Your CDN

Getting Drupal to play nice with your CDN can be a bit of a hassle. You have to make sure your assets (like JS, CSS, and image files) work not only on your webserver but when copied to the CDN, are served from there instead of your webserver. There is one Drupal module, the CDN module that attempts to make this a bit easier but right now, it’s not in production, and in my opinion, is a bit too complicated. There is a slightly easier way :)

Depending on which CDN you decide to go with you’ll want to make sure it offers HTTP synchronization. What this means is you don’t have to manually upload files to your CDN—if they don’t exist on the CDN, it will check your website for the file and download it through HTTP, putting it on the CDN. This works using some DNS magic.

For the ParentsClick Network we chose to go with Limelight Networks as our CDN provider. We configured out DNS so that we would have 2 static domains for performance (Yahoo discusses the benefits of 2 CDN domains): static1.pcncdn.com and static2.pcncdn.com. To roughly split which assets are served from which domain, we set anything that is CSS defined (whether the file itself, or any image defined as a url() within the file) as static1 and then any JavaScript or image as static2. We also chose a different domain for our CDN so that we could send assets without the cookie overhead.

When you request a file that isn’t on the CDN, it checks the fallback domain, which in our case, is parentsclick.com, and grabs the file from there, putting it on the CDN for you in realtime (in my tests, I didn’t even notice a delay, even using a 1MB zip file). This is all setup within our Limelight account.

So now that we’re sycning properly with the CDN, the next step is fix the URLs in Drupal. Of course, you don’t always want to use your CDN — especially during development, so you need a way to quickly turn this on and off.

First up, you need to patch Drupal, see the attached patch that fixes hardcoded paths for CSS, JS, and images. You’ll see that I’m using two defines: static1 and static2. In our settings.php, we added:

  1. global $enable_cdn;
  2. if ($enable_cdn) {
  3.   define('static1', 'http://static1.pcncdn.com');
  4.   define('static2', 'http://static2.pcncdn.com');
  5. }
  6. else {
  7.   define('static1', '');
  8.   define('static2', '');  
  9. }

We chose to use a global variable here. Your setup may be better off with a variable, a define, or something else. Then all you need to do is set $enable_cdn = TRUE to turn it on or FALSE to turn it off.

In my opinion, this functionality should really be baked into Drupal 7 and a patch woudn’t be too hard. Having a way to configure this under the performance section and being able to specify up to 2 domains to use would be great. If I have some time later I’ll work on a patch but if someone beats me to it, please let me know!

AttachmentSize
cdn.patch6.28 KB
posted 21 Aug 2008
  • drupal

16 comments

#1
Jordan wrote 44 weeks 5 days ago

This might be one of the best things ever done for drupal. I made one change, which was altering it to use link instead of the @import rules in your patch. Fantastic work. Thank you!

#2
ted wrote 44 weeks 5 days ago

@Jordan, yes on Drupal that is indeed a worth while change and luckily it is fixed in Drupal 6. Glad my patch could help ya out :)

#3
Siddhartha wrote 41 weeks 2 days ago

Hi, I saw that below links are same.
1) http://www.mothersclick.com/files/imagecache/group_logo_thumb/files/grou…
2) http://pcncdn.com/files/imagecache/group_logo_thumb/files/group_logos/30…
3) http://static1.pcncdn.com/files/imagecache/group_logo_thumb/files/group_…
4) http://static2.pcncdn.com/files/imagecache/group_logo_thumb/files/group_…

Where first two links are hosted on Joyent server and last two are on Limelight Networks for cdn (mirroring). But my question is how do you upload files on ‘no 2)’ that mean on ‘http://pcncdn.com/files/’?
My second question is if I have two servers one for application and other is for static file then how can I (or user) upload any image through image or upload module direct on the static file server?
Any suggestion?
Thanks .

#4
Irakli Nadareishvili wrote 41 weeks 19 hours ago

Ted,

auto-grab is awesome when file is new, but what happens if the file updates on the local server, how does that propagate to CDN? Obviously the file already exists on CDN, it’s just out of date.

I assume the “auto-magic” thing is not the only thing you rely on? Or they have something for that, as well?

cheers

#5
ted wrote 41 weeks 18 hours ago

@Siddhartha — we don’t upload any files. Limelight’s CDN checks to see if a file exist on their CDN and if it doesn’t, it fallbacks to our main Joyent servers and automatically uploads the files to the CDN. It does the heavy lifting.

@Irakli — yes, to expire content you can either rename files (cheap, CDN top practice) or set the TTL on the files to they expire automatically.

#6
Quinton wrote 40 weeks 6 days ago

Great post!

I have messed with this a bit myself using S3 and I find your methods insightful.

I was wondering, do you keep a copy of each file on the web server and the CDN, or do you remove them from the web server once they’re on the CDN? Since you’re using a fallback domain I would assume you keep a copy on both, but I just wanted to make sure.

Thanks,
Quinton

#7
ted wrote 40 weeks 3 days ago

Yes a copy is always on the webserver and CDN for backup/sync purposes.

#8
Drupal themes lab wrote 34 weeks 2 days ago

I think we don’t need to hack or patch drupal core, we need change url of js, css and others (see subdomain module did) and using rsync to sync files. I planned to develop a module support CDN. hope i will release soon to drupal.org

#9
david wrote 22 weeks 3 days ago

Hello ted, and thanks for the post !
It seems to be what i need, but i dont see the attached patch.
Have you removed it or is it just a bug ?

i would like to give a look at it and maybe use it on my upcoming website.

Thanks in advance

David

#10
ted wrote 19 weeks 1 day ago

@David thanks I moved the patch in an upgrade, it’s back now, enjoy!

#11
stuart wrote 16 weeks 20 hours ago

If you are having Drupal rewrite url’s for, for example, images to direct to the CDN, how does the CDN know where the original destination of the file is?

For example, if the file is originally located at http://www.mysite.com/sites/all/files/photos/photo1.jpg and with the patch it’s rewritten to http://www.mycdn.com/photos/photo1.jpg, when someone goes to my site for the first time, how does the CDN know to look at http://www.mysite.com/sites/all/files/photos/ to find the file and add it to the CDN?

I don’t doubt that it works; just trying to understand it.

Regardless, thanks for the great writeup!

#12
ted wrote 16 weeks 19 hours ago

Yes, when you setup your CDN (depends, not all support) you set a base URL to check if it can’t find the file. So in your case, the CDN will realize the file doesn’t exist, and then it will check your site with the URL you specified as the source. It will then grab it from there and sync it with the CDN.

It really just works :)

In my case I have used Edgecast and Limelight that supported this. I know others do but those are the only CDNs I have experience with.

#13
stuart wrote 16 weeks 9 hours ago

Can’t argue with something when it works! Thanks for the quick response.

#14
stuart wrote 15 weeks 5 days ago

Any chance you’ve updated your patch for Drupal 6.10?

#15
ted wrote 15 weeks 3 days ago

Stay tuned — I have a 2 new blog posts about CDN coming soon along with updated patches!

#16
Sandeep wrote 6 weeks 2 days ago

Thanks guys that was extremely helpful!

sandeep verma
(http://sandeepverma.wordpress.com)

Add your comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <p> <img> <pre>
  • Web page addresses and e-mail addresses turn into links automatically.
  • You can use Textile markup to format text.
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>. Beside the tag style "<foo>" it is also possible to use "[foo]". PHP source code can also be enclosed in <?php ... ?> or <% ... %>.

More information about formatting options


Code examples and downloadable zip files of code are licensed under a Creative Commons License.
All other content, unless where noted, ©2009 Theodore Serbinski. All Rights Reserved.