The following is a screen shot of a log from one client, who has a fairly large rack and a few servers for their thin client deployment. In this case the A/C failed and the servers shut down gracefully.

First noting the rising temperature and alerting CANDIS through our monitoring system and then when the failure threshold was met – issuing a shut down command to the OS and then powering off the server. All this happened on a long weekend and the final shut down was alerted to us as well.

Worked like clock work. Then again – I walk into many IDC’s in China and see flashing red failure LEDs and it seems they are forever on. Do SysAdmins in China not setup hardware monitoring? Do they not care? Why is this? Why would you want a failure to occur or interrupt your free time or your clients business?

Well it is not until you actually start to UNDERSTAND it – that you get to appreciate it. All too often the pundits of tech in society (as nice as they are), like twit.tv, cnet.com and zdnet.com – have a great habit of talking about tech and service – but never having actually done it themselves or can appreciate it at a deeper and more sophisticated level.

Case in point, is the fact that many people go on about “I want a real app, not some web app”. Or, I need to have connectivity while I am in a cross country flight, etc. To me it just seems that some people can’t get with the times from either a tech standpoint, or even from another perspective, say costs.

I am using software as a service to highlight the missed nuances of many a lay person or tech pundits analysis of such situations and trends.

I look at all the effort that goes into running email infrastructure, the costs of running servers and IDC’s. And it is quite common now for people to expect email to work like a light switch (major oversimplification) or to be a single small cost in the case of hosting.

Why is it, that software then and other technologies like thin clients, that seek to change the usage and costing of hardware and software from a disparate and ad-hoc, per person/cpu based exercise to a usage and deployment style that is more akin to hosting and email services, that one encounters so much resistance?

Resistance to change from clients and resistance to change from vendors. Why do people still see computing and laptops and applications as objects that can be traded and bought and sold like shells and clams?

I know I asked the question, but I can’t answer it. I simply can’t understand it.

Where we must give some props to in this regard is Amazon, Google and Microsoft, for somewhat seeing the light and being industry leaders. Amazon, offer pay as you need it grid computing power, Google offers online applications and now Microsoft is also offering online applications.

To be fair though, Microsoft has had licensing schemes in place like the SPLA model for a while. Allowing people to just pay a monthly fee to use all the software that they need. Never having to worry about upgrade costs or “upgrade insurance” as some snake oil salesmen like to push.

Allow me to further expand my thoughts on this, with respect to some recent dealings of mine and in China and some questions that I can answer! I will do this as a little Q+A session with myself.

Richard The Interviewer:

“Richard, why would you not want to own your own software and hardware and put it inside your office or in your lap on a laptop?”

Me:

“Thank you for the thoughtful question Richard, allow me to answer on the issue of hardware first. You see Richard, the problem with hardware is that, well it is expensive! And tis fact is made even worse by the fact that it is outdated and superseded so quickly.

Richard The Interviewer:

“But surely even if something loses it’s value it is still useful? I have seen many an old car drive around leaving a trail of rust behind it?”

Me:

“Yes, to a point. The problem is that software gets more complicated and our needs get more demanding as we become more used to and comfortable with our technology. A better way to look at it, is like hand-me-down clothes. Servers and infrastructure to a degree can just be demoted down the line with new beasts coming in at the top for critical systems and then older ones being eventually upgraded – or converted into a VPS as they come up for expiry.”

“But to answer your question more accurately, it has to do with technologies like Virtualisation, economies of scale, operational costs and idle capacity. If you look at your infrastructure like a time share holiday house or corporate jet, it suddenly becomes a lot easier to handle, both on cash flow and on your exposure to depreciation, so there really becomes very little in the way of real world value propositions to hold on to that old junk – this is even without going into the issue of power usage!”

“For many years people liked to finance their equipment. That is a great way to do it, as it lets you amortise the costs over the actual useful life of the system and then throw it out and get a new one. That way, you can look at the leasing fee as a perpetual subscription model. Just make a payment every month and the system is always online, up to date and fault free.”

“Plus it does wonders for your tax exposure when you claim all those leasing payments back against your tax. Much better than doing it the Chinese way with a 5 year depreciation curve!”

Richard The Interviewer:

“OK, I get it. Why own it and pay for the depreciation of it – that will take longer to remove a residual book value than it does to remove any operational value. And just pay one fee once, per month for life and always have the tech taken care of.”

“But it seems that if I do this, I am still having to manage the equipment and run an IT dept, even though I am leasing the hardware?”

Me:

“Yes, so why manage it then if you have already outsourced the ownership to your financing company? That is the strange thing that happens. It is like financing a truck and then paying for a driver. Why not just use DHL or FedEx instead. They offer economies of scale and are better at it.”

Richard The Interviewer:

“But I need control!!! I am worried about privacy!”

Me:

“Control = expense. Digital technology and servers = no privacy. Your needs to control and keep data secure are very valid. But to be truthful, simply having equipment on site or in your control does not make it any more secure or better. These are separate issues. And companies never really think about them any further than just that. I have it, so it is safe. I can touch it, so it is safe. This is really an illusion of security and control.”

Richard The Interviewer:

“Ok, so what about software then? Software as a service….?”

Me:

“Well software is no different. Take all that I mentioned above and change hardware to software. The one exception being that software in most countries does not have any depreciation allowance. In China for example, software is always booked in as a single transaction once into your overhead. That is the tax law. Even if the software cost 1,000,000,000,000 USD and you intend to use it over 20 years, you can’t book the expense in and match it over many accounting periods. So you are much better off in China at least operationally by renting or using an ASP service.”

Richard The Interviewer:

“So in a single paragraph, what is your main issue then with these old dinosaurs who want to own, control, touch and carry their technology around with them all the time?”

Me:

“Do I have to be polite? Well, I would simply say, “What are you thinking?!” Get rid of the big software investments and yearly upgrades and all that hassle. Just hand over a list of needs to an ASP, get a copy of their SLA (service level agreement) and be done with it. Problem solved – for life!”

“As to hardware, well, the beauty of an ASP, is that you don’t need any!”

Richard The Interviewer:

“Well thanks for that Richard. I think I will go and study the intricacies of ASP’s, ISP’s, ESP’s, the tax code, our cash flow, the tech road maps of the various companies out there some more. And work out just why it is, that I feel so compelled to continue handing over cash by the fist full to continue in this rat race of upgrade cycles, repair windows and deployment plans.”

“IT that is like watching a TV or renting a movie online…..hmm, that would be useful and stress free…..seems tomorrow has been here for a few years already!”

Me:

“Maybe next time we can touch on issues like data security, data retention, corporate governance compliance, audit trails, business continuity, capacity budgeting, staff retention, staff acquisition, staff training, staff vetting, staff knowledge pooling and on demand resources?”

Richard The Interviewer:

“That would be good. Is that a whole load of hassle too?”

Me:

“Well it used to be, but it is all par for the course when dealing with a good ASP, ESP, ISP or any xSP really. Even more value!”

Silicon Hutong

And the topic did strike a “Hey this is real man!!!” kind of chord with me. A cathartic resonance that shall never come from me with respect to the greater greenhouse effect crap while the infallible science is still not in. Nor came to me for any other “agenda” or “crusade of the world” that seemed to so very trouble all those students over in the Arts and Law faculties. That seemed to have so much time on their hands, relative to us poor saps in Engineering and Business school.

Politics aside, wastage for the sake of wastage though is not very good. And I have a rather poignant insider’s look at this issue that also manages (in my mind at least) to cross paths with this issue about China’s talent gap, or indeed this also about the talent gap.

What does resource wastage and power requirements have to do with a talent gap, or as they lament now in overly taxed and anti-entrepreneurial Australia – the “Brain Drain”?

Allow me to explain.

Servers cost a lot of money! Not only do you spend the 5-10K USD for the server and warranty, you then have colocation fees, cooling costs and power costs – this all adds up. And if you skimp on something like cooling, your equipment can age prematurely – especially bad in China given the tax depts allowed deprecation rates!

A good contact of mine who runs an IDC in New York city, mentioned how it is all about the power. Real estate? No worries, how much power will we need. So it seems pretty clear where things are going. Moore’s law, computing density increasing and an ever growing appetite for all things information.

With respect to the issues that David Wolf brought up, a major new trend that has been underway for the past 6 years or so and is trying to put a clamp on rising costs of infrastructure is Virtualisation. This technology allows for less resource wastage and consumables wastage by consolidating many lower end servers into fewer bigger more reliable and cost effective servers. Picking the low hanging fruit so to speak.

A further evolution of this that I mentioned about here is where the actual servers in a farm can be powered down too when not needed.

Now when it comes to electricity there are always two rates, the so called “Consumer Rate” and the “Industrial Rate”. And yes, data centre’s use the industrial rates, which while are relatively cheaper than consumer rates – are still not cheap in absolute terms. And if you go and use a nifty tool like this Dell capacity planner and then do some arithmatic, you can see what your costs will be. It is through this tool, that CANDIS deduced that paying the premium for higher efficiency power supplies and low voltage processors, actually did pay for themselves in outright power savings over a 2 year period. Even more if you factor in that less heat is better for stability and system longevity.

So where does the talent gap come into play? Well, in how many servers you need to do a job and knowing about virtualisation.

We had a client that was struggling to keep up and online with their 8 server cluster. They kept just purchasing new servers when things went bad and slowed down. Which seemed to happen all the time. Now I know this and they didn’t, is that an out of the box Linux system is not too tuned and the concurrent IP connection tracking vlaue in the kernel is a little conservative. So instead of them having 8 servers running at 20% capacity and still not managing the load on them as a cluster. Yours truly made a few changes and now 3 servers are doing the job, with better outward performance to the clusters users.

What made this situation all the more funny, is that in China, capital asset valuations are fair game for being compared to the salary of a team of workers to perform the same job (quality and reaction times not withstanding).

So, in summary, I will go out on a limb here and make a bold assertion.

“There are probably more servers than needed running in IDC’s across China, due to engineers not knowing how to admin them properly, not know how to architect systems properly and not knowing how to leverage new technology better.”

Anyway, I could go on. I just wanted to illustrate the nexus of those points and how they relate and intersect. Poorly deployed tech, poorly planned and architected tech and balooning operational costs and resource demands.

Seems that in the past you could be a lazy programmer and no one would notice due to the liberation of faster processors and free cycles. Or in China, buy a server is better than paying the for the expensive vendor support and optimisation.

However as one famous economist once mused…. “there is no such thing as a free lunch”

This place had official Olympic logo’s all over the place! And nice toilets – a rudimentary western yard stick when measuring quality in China.

This IDC has some big name sites in it. In front of our servers are some SOHU.com servers, Baidu was in the next room – past the hand print scanner – and sina.com was a few rows over too.

Great cooling, but still no sign of people getting “Hot Aisle/Cool Aisle” design. This IDC used overhead conduits instead of a raised floor. I guess it is all down to preference on that one. They do offer as standard two power circuits per rack, but only 16A per circuit. So forget about putting 60 blades in 6 chassis in these racks. One rack we saw already had an official warning on it about being careful when adding another server due to power use. Yet this rack only had 8 Dell PE2950′s in it?? Seems a bit wasteful on space to me.

Overall, very impressive. Apart from the hand scanners, the quality is the same as CANDIS’ current facilities. With some extras and minus some others. The good news is that most of what you would want is standard and then only a few missing things need be added. Where as we had to build a lot of stuff to get to the standard we wanted.

Oh well, in a city with as many cars and people as Beijing……location, location, location. We are in the CBD, in the embassy district. This place is out with the cows…… and no camera’s were allowed and I needed my passport and caused much fanfare being a “Foreigner” and all.

But in the end, I must say VERY PROFESSIONAL, as if anyone is doubting China’s commitment to out do Sydney (as evidenced by all my compatriots here doing a double take for Beijing) as the “greatest Olympics ever”.

CPU

System load was DOWN. This is due to much better disk IO (reduced wait times) and as such, user processes get to breath and send stuff back to clients in a speedy fashion.

LOAD

Load being a matrix balanced scorecard measurement over various metrics is also down considerably. The trend to take note of here is the 15min average more than the 1 min average. While spikes and load increases over short periods may upset some users experience. If the delays compound and extend over a longer time frame, you will be tying up more people in the slowdown. Again, this is leading a bit to the parallel vs serial arguments. And in the case of a database, sometimes fewer ‘in and out’ transactions give a better user experience over time because more gets done in a given time period, relative to many threads fighting at once. Think orderly lines at the bank in a developed country compared to the Beijing Subway at Jianguomen or Fuxingmen!

TEMPERATURE

A good 5 degrees lower in temp too. That is not insignificant. It all adds up to longer system life, better availability and lower costs.

POWER

Well out APC metered units aren’t that precise! While you can’t see any change from last week to this one – probably due to the load balancing effect of our redundant power supplies and redundant PDU’s AND UPS’ in between PDU’s and Servers (in addition to the IDC’s UPS’s and 4,000 Car Batts). You CAN see the increase in Saturday and Sunday while the massive disk arrays churn flat chat to reinitialise….that will need more power and also make more heat, needing more fan speed.