Its time for step two: to reduce internal bandwidth usage of our website by putting static content in the cloud.
I think I can do this fastest by using cloud servers, replicating our internal servers. I can do it better later using CDN.
I started a Red Hat Linux server on Rackspace in US. I installed Apache, MySQL and PHP. I configured the system like our own servers. After verifying that it worked as it should, I cloned it and started a second copy of it. Now I have two new web servers just like my internal ones, situated in the cloud in US. This was done in less than an hour.
I then started a Rackspace redundant load balancer for my two web servers. This is the first time I use such a load balancer, but it turned out to be very easy and done in less than 15 minutes.
The website had already been separated so that the more frequent images and some static content are referred by a different domain name compared to the dynamic website. Now its easy for me to change my DNS so that the static content is fetched from the cloud servers while the rest of the website remain on our own local servers. This needs some time to take effect.
After a couple of days I could once again look at the full page render times in our Apica monitoring system. This time I got 30% less download time from US, 30% worse download time from Sweden and no difference from Asia.
Once again, this result is rather expected since I have moved a lot of content from Sweden to US. But since 50% of our visitors are from US and only 4% from Sweden, this is far better for our visitors. This should however be improved by using a CDN.
Now I can also look at the internal bandwidth usage. It appears as if I have cut the usage of our local network with about 30%.