Popular content
- Leo talks about the new TWiT.tv site
- I love dirty tea
- 2 Guys Uncorked - Coming this January
- My first 15 min of fame!
- Rebuilding a BMW intake: S52 to M50 intake manifold conversion, a list of parts
- Mom Blog Network launches!
- SimpleMenu 3.0 released!
- Connecting Drupal and Silverlight
- New external links filter for Drupal
- American Legion Post 290
Getting Drupal to Play Nice with Your CDN
Getting Drupal to play nice with your CDN can be a bit of a hassle. You have to make sure your assets (like JS, CSS, and image files) work not only on your webserver but when copied to the CDN, are served from there instead of your webserver. There is one Drupal module, the CDN module that attempts to make this a bit easier but right now, it’s not in production, and in my opinion, is a bit too complicated. There is a slightly easier way :)
Depending on which CDN you decide to go with you’ll want to make sure it offers HTTP synchronization. What this means is you don’t have to manually upload files to your CDN—if they don’t exist on the CDN, it will check your website for the file and download it through HTTP, putting it on the CDN. This works using some DNS magic.
For the ParentsClick Network we chose to go with Limelight Networks as our CDN provider. We configured out DNS so that we would have 2 static domains for performance (Yahoo discusses the benefits of 2 CDN domains): static1.pcncdn.com and static2.pcncdn.com. To roughly split which assets are served from which domain, we set anything that is CSS defined (whether the file itself, or any image defined as a url() within the file) as static1 and then any JavaScript or image as static2. We also chose a different domain for our CDN so that we could send assets without the cookie overhead.
When you request a file that isn’t on the CDN, it checks the fallback domain, which in our case, is parentsclick.com, and grabs the file from there, putting it on the CDN for you in realtime (in my tests, I didn’t even notice a delay, even using a 1MB zip file). This is all setup within our Limelight account.
So now that we’re sycning properly with the CDN, the next step is fix the URLs in Drupal. Of course, you don’t always want to use your CDN — especially during development, so you need a way to quickly turn this on and off.
First up, you need to patch Drupal, see the attached patch that fixes hardcoded paths for CSS, JS, and images. You’ll see that I’m using two defines: static1 and static2. In our settings.php, we added:
- global $enable_cdn;
- if ($enable_cdn) {
- define('static1', 'http://static1.pcncdn.com');
- define('static2', 'http://static2.pcncdn.com');
- }
- else {
- define('static1', '');
- define('static2', '');
- }
We chose to use a global variable here. Your setup may be better off with a variable, a define, or something else. Then all you need to do is set $enable_cdn = TRUE to turn it on or FALSE to turn it off.
In my opinion, this functionality should really be baked into Drupal 7 and a patch woudn’t be too hard. Having a way to configure this under the performance section and being able to specify up to 2 domains to use would be great. If I have some time later I’ll work on a patch but if someone beats me to it, please let me know!
| Attachment | Size |
|---|---|
| cdn.patch | 6.28 KB |
18 comments
All other content, unless where noted, ©2010 Theodore Serbinski. All Rights Reserved.
This might be one of the best things ever done for drupal. I made one change, which was altering it to use link instead of the @import rules in your patch. Fantastic work. Thank you!
@Jordan, yes on Drupal that is indeed a worth while change and luckily it is fixed in Drupal 6. Glad my patch could help ya out :)
Hi, I saw that below links are same. 1) http://www.mothersclick.com/files/imagecache/group_logo_thumb/files/grou... 2) http://pcncdn.com/files/imagecache/group_logo_thumb/files/group_logos/30... 3) http://static1.pcncdn.com/files/imagecache/group_logo_thumb/files/group_... 4) http://static2.pcncdn.com/files/imagecache/group_logo_thumb/files/group_...
Where first two links are hosted on Joyent server and last two are on Limelight Networks for cdn (mirroring). But my question is how do you upload files on ‘no 2)’ that mean on ‘http://pcncdn.com/files/’? My second question is if I have two servers one for application and other is for static file then how can I (or user) upload any image through image or upload module direct on the static file server? Any suggestion? Thanks .
Ted,
auto-grab is awesome when file is new, but what happens if the file updates on the local server, how does that propagate to CDN? Obviously the file already exists on CDN, it’s just out of date.
I assume the “auto-magic” thing is not the only thing you rely on? Or they have something for that, as well?
cheers
@Siddhartha — we don’t upload any files. Limelight’s CDN checks to see if a file exist on their CDN and if it doesn’t, it fallbacks to our main Joyent servers and automatically uploads the files to the CDN. It does the heavy lifting.
@Irakli — yes, to expire content you can either rename files (cheap, CDN top practice) or set the TTL on the files to they expire automatically.
Great post!
I have messed with this a bit myself using S3 and I find your methods insightful.
I was wondering, do you keep a copy of each file on the web server and the CDN, or do you remove them from the web server once they’re on the CDN? Since you’re using a fallback domain I would assume you keep a copy on both, but I just wanted to make sure.
Thanks, Quinton
Yes a copy is always on the webserver and CDN for backup/sync purposes.
I think we don’t need to hack or patch drupal core, we need change url of js, css and others (see subdomain module did) and using rsync to sync files. I planned to develop a module support CDN. hope i will release soon to drupal.org
Hello ted, and thanks for the post ! It seems to be what i need, but i dont see the attached patch. Have you removed it or is it just a bug ?
i would like to give a look at it and maybe use it on my upcoming website.
Thanks in advance
David
@David thanks I moved the patch in an upgrade, it’s back now, enjoy!
If you are having Drupal rewrite url’s for, for example, images to direct to the CDN, how does the CDN know where the original destination of the file is?
For example, if the file is originally located at http://www.mysite.com/sites/all/files/photos/photo1.jpg and with the patch it’s rewritten to http://www.mycdn.com/photos/photo1.jpg, when someone goes to my site for the first time, how does the CDN know to look at http://www.mysite.com/sites/all/files/photos/ to find the file and add it to the CDN?
I don’t doubt that it works; just trying to understand it.
Regardless, thanks for the great writeup!
Yes, when you setup your CDN (depends, not all support) you set a base URL to check if it can’t find the file. So in your case, the CDN will realize the file doesn’t exist, and then it will check your site with the URL you specified as the source. It will then grab it from there and sync it with the CDN.
It really just works :)
In my case I have used Edgecast and Limelight that supported this. I know others do but those are the only CDNs I have experience with.
Can’t argue with something when it works! Thanks for the quick response.
Any chance you’ve updated your patch for Drupal 6.10?
Stay tuned — I have a 2 new blog posts about CDN coming soon along with updated patches!
Thanks guys that was extremely helpful!
sandeep verma (http://sandeepverma.wordpress.com)
Just an update, a better version of this patch is slated to hit core for Drupal 7. Check it out: http://drupal.org/node/499156
Hi Ted/et all, there is an easy way to do CDN on a drupal site which I blogged about a while ago. It kind depends on using simpleCDN or a network that allows mirroring via a URL. The lightning service on simpleCDN + the simpleCDN module do this and they are super cheap. The project and the service share the same name but are not associated.
There is my write up here: http://www.markparrott.com/blog/drupal-website-performance-improve-your-...
and the drupal projects here: http://drupal.org/project/simplecdn
and simpleCDN is here: http://www.simplecdn.com
Add your comment