Podcast transcript: Energy-Efficient Servers & Datacenters

August 27, 2007

Welcome to the ENERGY STAR podcast, brought to you by the U.S. Environmental Protection Agency. Thank you for joining us for this episode of our 2007 series dedicated to energy-efficient products and practices. For more information, please visit us on the Internet at EnergyStar.gov. That's EnergyStar.gov.

ANDREW FANARA: Welcome to another ENERGY STAR podcast. I'm your host, Andrew Fanara, with the United States Environmental Protection Agency's ENERGY STAR program. Today, with the help of a few industry experts, we'll discuss and explain the role datacenters play in our work and home life as our nation's computing backbone. We'll also discuss and explore the growth and demand for datacenter services. And lastly we'll talk about the implications for national energy use from the surge in datacenter services.

Have you ever thought about how your company stores data or what is powering the billions of searches each day on engines like Google or Yahoo!? From the smallest Internet startups to university laboratories, local municipalities to huge multinational corporations and federal agencies, people are relying on servers and datacenters to store and process massive amounts of information.

So with that, let me dive in and talk to our four experts that we have here today. First off we have Dick Sullivan, who's a Senior Marketing Manager with EMC Corporation; Peter Gross, the Chief Executive Officer and Chief Technology Officer for EYP Mission Critical Facilities; Christian Belady, a Distinguished Technologist with Hewlett-Packard; and Lorie Wigle, Director of Technologies Marketing at Intel's Digital Enterprise Group. Welcome to all of you.

LORIE WIGLE: Thank you, Andrew.

CHRISTIAN BELADY: Thank you.

ANDREW FANARA: Let's first start out with the most basic of questions. Maybe we could start with you, Christian, because HP has its fingers in many different types of businesses related to technology. Can you tell us more about what a datacenter does, what its functions are, what processes they serve in business and government, and what impact they have in terms of consumers' everyday life?

CHRISTIAN BELADY: Yes, Andrew. Well, as you know, HP is one of the largest providers of computer software and services in the datacenter. And if you look at what a datacenter is today, essentially it could be a huge facility — 50,000 square feet is not unusual — that essentially houses all of the servers and computers that we sell worldwide. And these facilities are critical to the operations of many organizations and really all organizations at this point. And they do everything from running the business, internal business processes, to running things, as you mentioned earlier, a lot of the Internet searches, billing, data mining, online banking, trading, the list goes on and on. But essentially every aspect of our lives is now touched by data that goes through datacenters.

ANDREW FANARA: So you might say it's the back office for the world.

CHRISTIAN BELADY: It truly is.

ANDREW FANARA: Peter, your firm builds datacenters for customers around the world. Tell us a little bit about some of the challenges there, and also if you could touch on what's different about datacenters today than, let's say, five years ago in terms of how they're being built and the demands for energy consumption?

PETER GROSS: I'm Peter Gross. I run an engineering firm specializing in the design and operation of datacenters. One way or another we are involved in a large percentage of the significant datacenters in the United States and, for that matter, around the world. The recurring theme we're hearing from our clients when it comes to the design of a new facility, of a new datacenter, involves issues such as reliability. That's probably the single most important component we hear. But more and more, especially in the last couple of years, the energy efficiency is becoming a significant driver. And interestingly enough, and probably we're going to discuss this later in this presentation, reliability and energy efficiency are not always convergent. Other challenges, especially in our world of continuous technology changes, refreshes, flexibility, ability of these datacenters to respond quickly to changes in technologies, power requirements, and cooling requirements in the size and weight and so on, are enormously important issues. Scalability, security, maintainability, ability to recover after disaster, all these are major challenges faced by datacenter owners and operators today.

As far as your second question, where is the datacenter industry different than a few years back, as I mentioned before, a few years ago the only driver, the single most important requirement as far as datacenters were concerned was reliability. Reliability was the element that shaped the form and design of a datacenter. Today, interestingly enough, energy efficiency, ability to reuse energy consumptions, primarily because of the costs and social responsibility, all these elements which were well described in the EPA document are changing the perception. And we see a greater demand, a significantly greater demand for high-efficiency data.

ANDREW FANARA: Well, you know, that's interesting that you mention that efficiency obviously has become such an important thing. And could it be, and maybe I'd ask Dick this, you know, it seems as though in our day-to-day lives that we increasingly have so many things that require tremendous amounts of information to be stored. And I think some of the obvious things that we hear a lot about are young people love their MySpace, and there's so much in the terms of music and entertainment that's out there, so much of that information has to be stored and managed. Can you talk a little bit about EMC's role in terms of where you fit in and what role you play in managing so much of that information that is proliferating?

DICK SULLIVAN: Sure, Andrew. One of the things that's interesting about the statement you just made is that, if you aggregate all of those different sources of data that needs to be stored, basically first it's created, obviously, it's captured at some — it's oftentimes replicated and then some places stored. In 2006, according to a study that EMC had commissioned, there were about 161 exabytes of data created. Take the problems that have just been described by my colleagues, and think of those in terms of the problem that we have right now today, and then think about what's going to happen as we have better than a 56 percent growth rate year over year for the next several years. We're no longer dealing in petabytes, which is a huge amount, but rather in exabytes. And so the amount of data that is being created and stored is something that is pressing not only today, but really in terms of how we plan for the future.

I think a major part of what EMC is looking to do is to make sure that, first of all, and I believe that this is not just EMC but really the industry as a whole, to look at a primary role at this stage of making sure that we've educated our customers and, quite frankly, even our own staff to see that they understand what the issues are involved. And that, while we are certainly looking at the development of new technologies, a significant amount of savings in terms of energy and even the amount of data that continues to be replicated and stored can be had by simply following best practices that already exist. The existing technology, if it is used properly, has the capability of reducing the amount of energy consumed, simply by seeing to it that it is applied in the wisest possible way.

ANDREW FANARA: That's interesting because I think it makes sense now to sort of turn it over to Lorie and have her explain a little bit about what Intel does, and their role. I guess, Lorie, your product, the microprocessor, is the brain of the server, or maybe you could even say the brain of the entire datacenter. You get both a lot of attention, both positive and negative, related to power consumption and efficiency. What's Intel doing to tackle issues of efficiency today?

LORIE WIGLE: So, Andrew, you're right. I think the microprocessor does receive a lot of attention when it comes to energy efficiency. For the datacenter overall, we kind of end up being the focal point. And Intel is taking that very, very seriously; and, in fact, kind of our mantra at this point in the microprocessor business is to deliver energy-efficient performance. And we think all three of those words are really important, especially the performance piece. It's really important that, as we focus on reducing energy consumption, we don't lose sight of the product that we're delivering, the cycles we're delivering to actually solve business problems and even solve climate change problems. So if you look at what we've done recently with the launch of the Core Microarchitecture, we made a very significant breakthrough in terms of both delivering more performance in our server product line, actually twice the performance, but doing that with 40 percent less energy consumption. So that's a really important piece of what we're doing.

But we don't stop there. The other thing that Intel is focused on is looking at the server platform overall and the other ingredients that go into the platform because they also consume energy. So we're working very closely with the industry. We're implementing better technologies for managing all of the components in the server and reducing power consumption when they're not in use.

ANDREW FANARA: You know, I hear a lot about, Lorie, the new designs for servers which have many, many more processors and cores in them. Can you sort of explain in layperson's terms what that means in terms of four cores and what it means for performance?

LORIE WIGLE: So what Intel has done over the course of the last few years is, rather than just increasing the performance of a single core or processor and continuing to deliver, again, more performance that way, we've actually realized that we can both deliver more performance as well as reduce the energy consumption by instead multiplying the number of cores. So we went from one core to two core, and now our highest volume processors actually have four cores. And again, that allows us to deliver more performance in a lower energy envelope. It also has implications, though, for the software industry to make sure that they have written their software to take advantage of that. And it's also been one of the really strong enablers and drivers of virtualization software because it is designed to inherently take advantage of multicore technology.

ANDREW FANARA: Well, let's take a step back before we jump on that issue of virtualization a little bit, because I do want to discuss that, and sort of move on to the basics of the report that EPA has written that will be sent up to Congress hopefully very soon. The report that we've written basically concludes that we believe we can increase performance without sacrificing product efficiency or uptime or all those other important criteria; and efficient technologies and practices could improve efficiency of servers and datacenters considerably, somewhere between 20 and 55 percent.

And I know Dick mentioned best practices, as well, and I'd like to sort of touch on that, if I could. Christian, what do you think in terms of those estimates? I mean, HP does many different things in a datacenter, managing datacenters, building their own datacenters, in terms of selling lots of products and integrating all of them. In terms of best practices, do you think those numbers are realistic?

CHRISTIAN BELADY: Absolutely. So across the board, you know, we believe that simply, with just simple things, you could substantially improve the operations of your datacenter. And, you know, the report, the EPA report does address many of those things, which is great. In addition, there's technology that's emerging that can also significantly improve efficiency, power management technologies, you know, of course everyone on the phone is a key player in making those things happen, from Intel, EMC, HP, all of us really have been looking at really limiting over-provisioning. When utilizations are low, you slow down your — you throttle back on your processor, for example. And so there's a lot of things that technology can do.

But even at the datacenter level, while, you know, folks like EYP do a fantastic job in building state-of-the-art datacenters, I still think there's a problem with implementing to the original intent of those designs. And so I think there's a lot of best practices that, you know, we've been sharing, and of course the industry's been sharing, on how to get the most out of your datacenter. So clearly there's lots and lots of opportunity. And, you know, while many people think that this is a big problem ahead of us, I actually see it as a big opportunity for all of the players involved in this industry.

ANDREW FANARA: Well, Dick, maybe you could tell us a little bit about virtualization as one of those technologies that has so much promise and has been talked about so much. And maybe you'd explain what it is and what it might do for people. I know that your company, EMC, and VMware is a big player of that. Could you explain that to us?

DICK SULLIVAN: Sure. Essentially, the most typical way that servers have historically been used is as an application becomes important to a company, they will create or designate a specific server to run that application. And one of the reasons why there has been such an increase in the amount of energy consumed is because there's been such a proliferation of the number of different applications that businesses now require, particularly driven by needs that result from putting these on the Internet. As a result, there's been not only a proliferation of servers, but oftentimes those servers themselves are underutilized. In other words, the total capability of a server is much higher than the actual utilization rate. It's not uncommon to see something as low as 10 or 15 percent. And you'll see variations in those numbers. But I think in general people agree that it's much lower than it could be.

Essentially what virtualization does is allows software to take what is typically a more powerful server, but then make it look like it is actually multiple servers, so essentially virtualize those physical servers through software so that they are represented as separate virtual servers on a single large server; therefore essentially say I can unplug multiple servers and combine them into a smaller number of powerful servers. We have seen that happen in a number of instances where customers are able to reduce by very significant numbers, you know, sometimes — we have an example where a company had gone from nearly 1,200 servers down to something under a hundred, and saved nearly 80 percent in terms of the amount of energy they were using, at the same time increasing their utilization to as much as 80 percent.

The basic idea is, I'm having few physical servers, typically more powerful servers, and then virtualizing the information in a way that allows me to manipulate it in a much more simplified way. And oh, by the way, the benefits of virtualization are much greater than simply energy efficiency. But in this instance, energy efficiency happens to be a significant additional benefit that you get beyond the benefits that come from virtualization.

ANDREW FANARA: Well, I'd like to ask anyone to jump in on this question. And, you know, the common reaction from industry oftentimes when they heard that EPA was interested in studying datacenter energy use was why does EPA care about datacenters? We believe the EPA, and in conjunction with the Department of Energy and the federal sector in general, can be a catalyst to encourage better efficiency. But can anyone sort of jump in and sort of articulate what they think the federal role should be? Or maybe where we shouldn't tread? Anyone want to take that?

PETER GROSS: Well, this is Peter. Probably — I read the report, and I think it's terrific. I think that developing a framework, a roadmap for the industry to follow in order to improve energy efficiency is the single most important element. Probably, and I tend to assume that most people agree that what we lack in this industry more than anything else is a set of matrix, a set of matrices that will help us really quantify performance in many respects, but specifically in terms of energy. And I think that the EPA document has helped tremendously in bringing all large companies and groups together using a single fundamental matrix, the PUE, which is, for better or for worse, is what we have right now. And I think that can have the role of unifying, converging all efforts in the direction of improving that. At least, at least if nothing else, the role of this document and the EPA in general has been to bring some focus on this issue, making the world understand how critical is energy reduction in datacenters.

And, you know, simply the datacenter might not consume such an enormous amount of energy. But if you look at the concentration, the visibility of a datacenter, a large datacenter today, not a typical, but a relatively large datacenter uses more energy than a city with 25,000 people or so. And these are staggering numbers. This is one building with a load of, let's say, 50 megawatts. And if you take into account all the power and cooling required to supply electricity to these computers, you are talking about 25 megawatts. That's the power required by a good-sized town. And if you take into account the energy, it's certainly a lot more than what the town will use. So there is a clear need for a guiding document, at least, and some direction in bringing these divergent efforts in the world towards improving these datacenters.

CHRISTIAN BELADY: This is Christian. I completely agree with Peter. One of the things I've seen happen, while many companies like HP have been doing a lot of work, we've been doing a lot of work around efficiency computing for about a decade now. But the industry has still been rather fragmented. And just the idea of efficiency and the notion of efficiency wasn't brought really to the forefront. And what I see the EPA doing was it really helped facilitate people coming together, just as Peter said, to work together, both directly — the EPA has influenced this — and indirectly, you know. Directly when they sponsored the first meetings back in January of 2006 and brought together competitors and partners in the industry to kind of come up with how do we go from here and try to drive alignment, which was really a great first start. And then there was the second meeting, I think it was April of that same year. And since then you see the emergence of this report. You've seen a lot of work about coming up with metrics, and Peter mentioned some of those. But included in that is now we are seeing a server benchmark will be emerging in the next few months.

And then of course there's the indirect effects. Now there's a lot of industry initiatives have been kicked off and consortiums such as the Green Grid, as well as Climate Savers. So the EPA's role has been really kind of, as you said, Andrew, kind of the catalyst to kind of get everyone to play together. And it's been a critical role. And I think right now there's, like, this good critical mass to move things forward and drive towards efficiency.

ANDREW FANARA: It's interesting you mention Climate Savers. Lorie, I know that Intel was very central to getting Climate Savers, the Climate Savers Initiative, up and running. Can you talk a little bit about that and what the interplay might be with respect to datacenters?

LORIE WIGLE: Yeah, thanks, Andrew, for bringing that up, or letting me talk about it a little bit. What the Climate Savers Computing Initiative is focused on is actually some relatively straightforward things that companies and individuals can do that will have a really big difference in terms of the energy consumption of computers in particular. And there are really two facets to the program. One is focused on desktops; the other is focused on servers. And within each of them, what we've done with the Climate Savers specifications is outlined energy efficiency targets for just the power supplies that are used in these systems. So it's, again, a pretty simple approach, also very aligned with the EPA's ENERGY STAR program, so that we're all asking people to implement and to buy at the same level.

And then the other element of Climate Savers which is extremely actionable is for, again, individuals and companies to implement power management. And often what we find is, although we have good power management technologies shipping from the manufacturers on systems, it either gets disabled in the BIOS of a server, or in a desktop it might get turned off, or there might be some software that disables it. And we're just — we're giving people through Climate Savers the tools to make sure that they're using their power management capability properly.

ANDREW FANARA: Well, you know, we certainly talk about trying to encourage, at the EPA but elsewhere, efficiency and sustainability from the desktop to the datacenter. What do you envision as sort of key milestones going forward for this initiative, Lorie?

LORIE WIGLE: So the key milestones, actually on an annual drumbeat we will release new targets for the efficiency element, starting first, as I mentioned, with power supplies, but evolving that to also be the motherboard or baseboard in the platform. And then we're working actually very hard at this point to enable some tools or distribute some tools for power management that will be available on the Climate Savers website. So those are a couple of very near-term actions that we're taking. When we launched the program, which was actually on June 10, we had 40 companies signed up for it. And we've already doubled that. And we're going to be doing a lot of outreach to continue to grow that. And hopefully you'll be seeing some announcements from us in the near future about how the program's ramping up.

ANDREW FANARA: Well, I know we've talked a lot about in the last 30 minutes many of the opportunities that seem so tangible to all of us in terms of implementing better efficiency in the datacenter space. But we know that there are lots of institutional barriers in organizations to implementing energy efficiency strategies or energy plans in an organization. Has anyone out there seen companies who have done a particularly good job of this at the corporate level?

DICK SULLIVAN: Andrew, this is Dick. And I'd like to comment on that from a couple of perspectives because what I've seen is that there are many companies that at least are waking up to the issue; whereas perhaps even as short a time as a year ago, energy was something that was totally beyond even their peripheral vision. It's come very much into focus in the past 12 to 18 months. And they begin to understand that they need to have something that goes beyond simply a tactical way of dealing with the current energy problem because they have a piece of equipment that they need to plug in that's just arrived on their loading dock, and are beginning to grasp the idea that having a much more robust energy strategy is something that they need to do almost as they do a financial budget, on a quarterly and annual basis, to understand where they are, where they're going, and what strategies they need to implement in order to get there. And I think that more and more people are waking up to that.

That said, I think that there are some that actually have done an admirable job. Citigroup is one that has not only appointed some senior people to be responsible for looking at energy efficiency and sustainability overall, but actually have not only looked at the datacenter per se, but looked at educating their own staff as to how to be more energy efficient, are using this as an opportunity to save energy, not only in the datacenter, but on a corporate basis in local offices by being more efficient with everything from lighting to air conditioning. And so they've taken a much broader approach, although the energy efficiency in a datacenter is an important aspect of it, to make everyone in the corporation more conscious of the need for energy efficiency.

ANDREW FANARA: Dick, I want to return to something you said there, and anyone can chime in on this. But increasingly, organizations, corporations around the world are looking at energy security issues, energy supply issues. Also laid on top of that are issues related to climate change. Datacenters clearly for many organizations are large consumers of energy. In spite of the tremendous benefits they provide, they do consume large amounts of energy. The cost of that energy is getting more expensive in general. How would — or do you think that managing your datacenter in a more efficient way can be part of a risk management strategy for any organization, or should be a foundation for a risk management strategy?

PETER GROSS: Yeah, I think it is. I think the corporations today are looking at the issues of performance, performance in terms of reliability or availability, or in terms of security, physical security, cyber security, flexibility, all these elements are looked in conjunction with efficiency, energy efficiency. You asked the question earlier about the barriers some companies may see in implementing some of the efficiency measures. And as I said, almost without exception, every single corporation we work with today has in place some sort of an energy reduction program. It's amazing to me to see how quickly companies in this country, and to a certain degree around the world, but primarily in this country, have adapted to these major trends. And everybody is looking at reducing energy.

The problem is, especially for large transaction-based institutional datacenters, where there is a perception there might be a conflict between reliability, the way we look at reliability today, and energy efficient. You know, high reliability translates typically into a high redundancy level, and high redundancy level means lower loading, which is directly correlated to efficiency. So high reliability might not necessarily result in better efficiency. So there is a conflict, although there is a tremendous amount of work today to bring these two components together.

One of the things we have been doing recently is developing some matrix ways of correlating the costs of the facility and the energy consumption and reliability, finding a way of analyzing and correlating these three components — cost, efficiency, and reliability — and optimizing the performance of the facility to achieve the best performance.

ANDREW FANARA: Christian, what about you, when you are visiting customers, do you increasingly see efficiency as something that people see as an opportunity, or something that they are leery of? Do most folks rely on sort of a business-as-usual perspective when designing and operating their datacenters? What do you see out in the field?

CHRISTIAN BELADY: We see both. Clearly there seems to be a strong correlation with what energy costs are in that region. You go over to Japan, and they're extremely concerned about energy costs and much more aggressive in terms of looking at virtualization to make the power footprint smaller in their datacenters, as Dick was mentioning earlier. But in some parts of the world where energy isn't as expensive, people are a little bit more relaxed, and they're less likely to take risks. People are really trying to optimize between those three components: risk, cost, and efficiency. But at the same time, I think also, regardless of where you are in the world, efficiency — we're at a state now where I think there's a lot of low-hanging fruit. And the report does cover some of these things, where it doesn't necessarily mean that you have to give up on reliability, doesn't mean you have to give up on performance. Because we're still kind of in this fragmented world of before they even started building the Model T in the automotive industry; right? Cars then weren't that reliable. But since then automobiles have become quite reliable, and I think without giving up performance. And in fact, they're performing better.

So what I see is that, if we really look at what the opportunities are in the industry. we may not have to give up anything from the standpoint of risk, performance, costs. In fact, we'll probably do better on costs because year over year performance is better for the processors that Lorie was talking about. And ultimately we can kind of peel away that outer layer of inefficiency and not impact anything else. That's our first task at hand is to get that first layer.

ANDREW FANARA: Well, to tackle that problem, Christian, do you think that — and actually we make this point as a key recommendation in our report. You can't improve something until you can measure it and create a standardized way to measure something before you would actually see real improvement from some of those best practices. You've talked a lot about a ratio used to compare the amount of energy consumed for a datacenter versus what is consumed by the information technology equipment — the servers, the storage, the network, the routers, et cetera. Talk a little bit about how that could be a springboard, or at least a metric for that could be a springboard to really encouraging or giving organizations something to shoot for when they are putting together a plan.

CHRISTIAN BELADY: As you know, I'm very passionate about that, for the reason that I've been in situations with customers where they did all of the best practices, they improved things; but they were concerned that the hot aisle was too hot and uncomfortable to work in, and yet all the server temperatures were still the same. They went back and said, you know, it's uncomfortable, we're going to move everything back the way it was. And their whole problem — and I was devastated because their whole problem was that they could not measure the efficiency improvements in their datacenter. If they would have recognized the efficiency improvements in their datacenter, they would have understood that this is a good thing.

And so I've been real vocal about — and I think it was Peter that mentioned this earlier about efficiency metrics like the PUE and what you actually listed, the datacenter power use over the critical load use. The reality is, I don't care what you call it, but we need to be implementing that kind of ratio today so we could compare technology, datacenter technologies against each other. Datacenter improvements year over year, when you build your new datacenter it should have a better ratio than the previous year. It's absolutely critical to measure these improvements because otherwise it's just hearsay.

LORIE WIGLE: Well, I think, Christian, in addition to the measurement so that people are really tracking the efficiency, having a measurement that spans both facilities and the IT equipment causes conversations to happen amongst the various constituents in IT that might not have otherwise, which is also very healthy, I think.

CHRISTIAN BELADY: Yeah, I agree. And, you know, in particular, we talked about this a little earlier, there's the server benchmark that SPEC will be releasing, as well. And that's, you know, we have a datacenter metric possibly, and then now also the server metric. And it creates a dialogue, you're absolutely correct.

PETER GROSS: Yeah, I couldn't agree with you more, Christian. It's important to understand, in my opinion, that efficiency improvement is a collection of steps. I don't think there is one single step that would ultimately address that. We need to look holistically at the datacenter from end to end. And in order to really achieve a significant improvement here, one has to look at the five main elements in power usage in a datacenter. And if you, you know, not till they are the primary energy source, which typically doesn't have a lot of ability to control, it's the power, the utility coming in the building, although there is more and more interest right now in cogeneration so that some of the transmission losses can be reduced.

But then you look at the electrical distribution systems for the facility, the mechanical, the HVAC performance, server performance, and the IT performance. And if you analyze each of these five elements, and you identify the steps that can be taken quickly and easily, and other steps that require some work and investment. But ultimately the key is to identify all these opportunities, do an economic analysis, and determine which step will have the highest impact; and then develop a comprehensive plan, step by step, improving the overall performance.

CHRISTIAN BELADY: Exactly, and — this is Christian. What I'd like to add is that those steps aren't necessarily the same for every business and every industry. But at least those are the components.

DICK SULLIVAN: Hi, this is Dick. I'd like to add something to that, as well, that I think you're absolutely right, those steps are different for each industry. Andrew, you had asked a question before, what is the reason for the EPA to be involved. I think in many respects it is, number one, as a source of education, basic information about what the issues are and what some of the potential resolutions to those issues are. But I think that there's another, and that is that, as we talk about what can happen to an individual datacenter, if you imagine a city block in a dense population such as New York City, if we have two datacenters that do the right thing and five that don't, potentially even the two that have done the right thing will be impacted by the five that did not. I think that it's important that there be an understanding that these are things that need to be, not only promulgated in an individual datacenter, but across datacenters; and that this is not something that happens only in North America, but something that really needs to be addressed globally.

ANDREW FANARA: Well, on that note, I'm glad you mentioned the global issue because many of you spend time outside of the U.S. Is there anything that really differentiates datacenters in the U.S. versus their counterparts in Europe or Asia or anywhere else, for that matter? Can all of the things that you're talking about be implemented anywhere? And could governments across the world and in different places that are trying to encourage efficiency emulate some of the recommendations that EPA is thinking about or putting in their report?

DICK SULLIVAN: Well, I think that there are two things. One is that, in general, most of the best practices are best practices per se and have no geographic boundary to them. I think that the other thing is that we have to understand that all of the companies that are represented on this call, and certainly many, if not most of our customers, are also global companies. And so as we look at implementing best practices, we don't want to do something in the U.S. and then not do it in Europe or in Asia. And I think if we look at our customers in those other countries, in many instances they are looking to us to help them understand what the best practices are.

Now, of course there will be some differences. The amount of geography available, for example, in the U.S. in order to create a new datacenter is quite different than what's available in Europe or in Tokyo, say. Density will be an issue in some instances. But the general best practices in terms of virtualization, which we talked about earlier — and by the way, I should also add that virtualization is something that includes storage, as well, and there are ways of virtualizing and basically consolidating storage that helps to reduce the overall amount of data or the amount of storage that's required. But these best practices clearly translate across borders.

CHRISTIAN BELADY: Yeah, and this is Christian. What I'd like to add is I think it's extremely important that we try to influence the rest of the world also to look at adopting some of the recommendations that's in the report because one of my concerns always is, you know, I'm a firm believer in standardization. The last thing we want is a fragmented globe in terms of having different energy recommendations on what to be doing, and that creates confusion, and then a lack of adoption will occur. So what I'd like to see is kind of a global type initiative that kind of aligns all of the various regions of the world in following a similar type of best practices, using similar metrics, and cause kind of a very aligned and kind of almost a strategic type of direction for efficiency in the industry.

ANDREW FANARA; Well, I think on that note we are going to wrap up. I want to thank all of you. And I think we've all benefited from your insights and perspectives. And I hope that you'll come back and talk to us again sometime soon.

Thank you for listening to the ENERGY STAR podcast, brought to you by the U.S. Environmental Protection Agency. To hear more about energy efficiency, at home and at work, subscribe to our podcast on iTunes or keep checking EnergyStar.gov for more episodes. Thanks again for listening and for your interest in ENERGY STAR.