Today we are excited to announce we have rolled out a significant improvement to our backup scheme deployed on our cloud platform. Our platform runs a hardened Linux stack using KVM virtualisation technology which we have optimised specifically for running our Supportworks ESP software. One of the capabilities this gives us is the ability to take on-the-fly snapshots of running images without service disruption – this capability formed the basis of our initial backup scheme. While this is an easy and popular scheme to deploy it actually has some drawbacks which we wanted to address. Images can be large; some of our customers have upwards of 150-170Gb images so moving these around is time consuming and resource hungry. Ad-hoc file restores are a real pain and are only possible with manual intervention, but perhaps the most important issue is that of snapshot frequency, which we currently do once per day, once per week and once per month in accordance with our SLA. The problem with this approach is, if we did have a catastrophic failure and had to rely on a backup snapshot then depending on the time of day we could lose data which we felt was totally unacceptable. Of course we have High Availability N+1 redundancy on each POD at all times so this is a very unlikely event, none the less a possibility – for example, a natural disaster that affects one of our primary data centers could produce this outcome. To mitigate against this possibility we have developed and rolled out a new and improved backup scheme across our entire cloud platform and all customer production instances.
The new scheme uses real-time replication of both database and file system data using a pull replication technique that works on data set delta’s – making it very efficient. This will not have any material impact on performance or resource utilisation on our customer’s production instances. Data from each customer instance is replicated to a secure secondary data store located in a separate physical data center in the same geography but at least 100 miles from the primary data center. We expect replication to be near-real-time for SQL data and a maximum tolerance of just a few minutes for other data such as file attachments and documents, with future product enhancements that will drive this time window down even further.
Customers can log into http://trust.hornbill.net/ and see a real-time status of their instances which now also includes a real-time status monitor for the backup activity. With this now in place customers can relax in the knowledge that their Supportworks ESP instance data is now safely and securely stored in two physical locations at all times ensuring service can be resumed quickly should any disaster in one of our primary data centers strike. The system paves the way for some exciting new features of the platform coming soon including automated data escrow, scheduled and customer requested snapshots and on-demand sandbox instance creation for development and testing purposes.
This new capability is provided to all of our existing and all new cloud customers, there will be no service disruption and you do not need to do anything to take advantage of this, the service is already enabled and the service is provided without additional charges.
Thank you for being a Hornbill cloud services customer.
How many times have you set up an account on line and had to think of a password? Your bank needs one, your email needs one, your Facebook account needs one. Passwords are everywhere: your bicycle D-Lock (no relation) requires a four-digit password. How many of you reuse the same password across different services to avoid having to remember too many passwords? Of course you never write these down, SMS them to yourself, email them to yourself, write them down on a scrap piece of paper, or share them with a parrot, do you? I’ll blog about password storage another time, just think on in the meantime about how vulnerable your passwords are.
For now, though, I want to focus on how secure your password is by looking at password strength. Password strength is an indication of how easy your password is to hack. You’re probably thinking it’ll never happen to you. Can you be that sure? Really? There are people out there just waiting to get their hands on you account details, to steal your money, to send loads of spam email originating from your actual account, to post embarrassing status updates and pictures on your Facebook account, to lock you out of your account. By way of confirmation of how easy it is to hack a password, have a look at the site http://howsecureismypassword.net/ (I wouldn’t recommend typing your actual password in, just in case) The site also tells you many people choose particular words as passwords. Yes, I was surprised that particular word was among the 300 most common ones chosen as a password!
Back to the serious matter of securing your password, it is important that you choose a password that is strong. Some companies have a password policy — a topic of a future blog post — that forces you to choose a strong password. If your company does not have a policy, or the policy is not adequate, then you can strengthen your password by:
- avoiding the use of words
- using a mixture of upper case and lower case letters
- using special characters and numbers
- ensuring the length is at least 8 characters
We have come full circle, in that the more complicated your password is, the harder it will be to remember. Paddy can’t solve all your problems in one post 😉
The takeaway message from this blog is: don’t make it easy for the hackers — choose a strong password….Please!!
There are no maintenance events scheduled at this time.
We are pleased to announce the addition of a new data center based out of San Antonio, TX from which we are now providing our full range of cloud services including http://myservicedesk.com and http://hornbill.com. The data center is one of 17 SSAE 16 Type II certified facilities operated by our dedicated hosting partner Peer1 (http://www.peer1.com/infrastructure/datacenter-sanantonio). The new point of presence will service the growing demand for our cloud services in North America providing low-latency cloud services from the south central region.
You may have experienced some intermittent problems connecting to your instance around midnight. The on-call team were alerted to the problem and they promptly established that there was an issue at our UK datacentre. A call was logged with Peer1, who confirmed that there were network problems in their Portsmouth datacentre. The intermittent problem lasted just under 10 minutes. We will continue to monitor the situation through the night.
At this time, all network access to the Portsmouth DCO has been fully restored and we are not recording any further packetloss or latency. A full post mortem will be made available in the near future. If you are still experiencing any connectivity issues, please reach out to support immediately. We apologize for any inconvenience caused.
Today I felt like a real celebrity: I tried to set up a twitter account and found that my identity had been stolen. There is another Paddy Lock? Surely not. Who is this @PaddyLock?? It’s not me. I did a bit of digging and found that this is the Face of The Total Security Summit at Forum Events Ltd. How dare they?! I have a face, two arms, two legs and am anatomically complete! I am a real person, so in true “celebrity” style have set up @RealPaddyLock. Do follow me.
Beware of imitations!!
P.S. http://www.facebook.com/people/Paddy-Lock/701154141 isn’t me either
This week I was experimenting with locking down Apache to avoid a particular vulnerability that a hacker may want to have some fun with. With a bit of research, I found a very cool third-party firewall for Apache that can be used to close this vulnerability — I’ll blog about this firewall another time. We built the module from source, installed it and then applied the lock down…I couldn’t wait to try it out….I tried a very simple example of the vulnerability, the simplest I could think of. IT DIDN’T WORK!!! What did I do wrong? I checked, double-checked and triple-checked the configuration applied to prevent the vulnerability: there were no errors my side. With a bit more research, I found out that there was a problem: the latest release of the third-party component did not prevent the vulnerability, although previous releases did. The latest version had regressed 🙁
Oh well, these things happen. The third-party component is great and really useful, so I can live with reverting back to a previous release to achieve my goal. The whole episode did prompt me to write this blog post and provide some words of wisdom that others may benefit from:
- always conduct your own tests to make sure that a third-party component does what it says on the tin
- create a regression test suite to make sure that old problems are not re-introduced when you upgrade a component
- don’t assume that security experts will not make mistakes!!