Security experts Chris Hoff, Rich Mogull and Dino Dai Zovi talk about the organizational challenges of virtualization, including the most dangerous way to use virtualization in the enterprise.
Also check out:
- Part 1: Greatest threats to virtualized environments
- Part 2: Security benefits of virtualization
- Part 4: Virtualization and security vendors
Read the full text transcript from this video below. Please note the full transcript is for reference only and may include limited inaccuracies. To suggest a transcript correction, contact firstname.lastname@example.org.
Virtualization: Disruptive technologies part 3
Eric Parizo: So there is organizational challenges that you
guys mentioned.Does a lot of that boil down to just sort of
fundamental lack of understanding about the technology itself?
Rich Mogull: Think about how stuff gets picked up. I know from when I
first looked at virtualization I was still in my sys-admin days when we
looked at that it was kind of like well I have this particular problem as
an administrator of this server. If I could use VM and I could start
moving my stuff around and managing it at that level and you never
think to talk to the application owner.
Dino Dai Zovi: You don't need to.
Rich Mogull: Right, they don't know what's going on.
Dino Dai Zovi: That's one of the selling points.
Rich Mogull: So I don't think it's like, pardon me if you think I'm
wrong, but we don't understand the technology or anything else it's just
that the way you think if you're the sis-admin is managing that system or
that application admin or whoever's got the responsibility for that. You
don't really think about telling anyone else because it doesn't' affect how
they do their job, in your mind. And then it turns out later it does.
Dino Dai Zovi: I think the big political challenges are turf battles
basically. With virtualization, virtual switches, with virtual all these
other things. You don't have anyone who needs to know IOS, based on
Cisco technology so all of the sudden basically the network team
feels very threatened because their turf is now being encompassed
or encroached on by virtualization. Same thing happened to
database administrators with a lot of the high level object relational
mappings and application layer. Developers loved it because they
could change things without requiring big database changer
procedures. It's the same thing with virtualization.
Chris Hoff: On the security front to. The reality is if you look at
what security means to a virtualization platform like the VMware today
without the, let's say the API's it's a couple put down boxes to prevent
max- booting, and retransmits, and for transmits etc, etc. And the rest of the
documents all point to how to secure this service counsel, that's security.
The reality is there's a ton of fud about how you can use guest-to-
guest, or guest-to-host, or guest-to-hypervisor, you can do all
these attack scenarios that can boil down to productions that are
ridiculous to where you never deploy them in those states or in many cases
you'll find people talk about hosts of virtualization platforms with
features that are meant for a desktop enable certain features and functions
that are not distinguished between host and server based platforms or
hardware, so people knee-jerk and jump all over the place.
Security teams don't have the visibility into the tool sets to mange this
stuff because they don't have access to the management tools so they get
wound up. Right now this market is starting to settle down. Because what
you're finding is the virtualization platform provider has actually
realize we've really caused a scene out of West-side Story. I mean they're
sitting there with switchblades, and they realize that in many cases the
security teams are exercising veto powers. Same thing with the network
teams where they're starting to throw up roadblocks right, wrong, or
indifferent, but it's happening because they're not included in the
process. For whatever reason. But I think what's really interesting is
that when you talk to folks who are implementing virtualization, they have
the opportunity to but are somewhat discouraged sometimes to say, well we
want to do it right this time. The server admins, network, and security
the want to do it right this time . But what that really means in many
cases, without having to turn your existing network upside down which
really breaks the whole utility model of virtualization, what they end up
doing is replicating what they already have. And then we're back at the
cycle of we'll bolt on security later. So then people are starting to look
at emerging tools, taking what they already have in the physical world, and
trying to make it run as a virtual appliance. Rich was alluding to my Four
Horsemen of the Virtualization Apocalypse. One of the topics is that you
can't actually replicate because the software model for that hardware
appliance doesn't really adapt itself very well in the virtual world. It
doesn't scale, you can't do HA. To force data through a virtual appliance
like a firewall within a virtual machine environment means that you're
actually re-architecting the network. And yes it's easy in the virtual
world but what that does is dramatically impacts for example what happens
up the chain with ECO rules and firewall rules in the physical side. So
there's an awful lot of turning going on right now in that market.
Rich Mogull: Not to mention just the performance penalties of that. I
mean the big thing to think about is virtualization is a tool like anything
else there are places where it excels, and where it excels with your
current management techniques and current tools, use it by all means.
Areas like running a firewall in a virtual appliance in pure software on a
hypervisor. There's also crossing two security boundaries, that's probably
the worst place possible to use virtualization. Anyone who uses a virtual
appliance firewall is just asking for problems.
Chris Hoff: That's the topic that I talk about. The optimal
vitualizations say, test and depth, they may in the physical world segment
their networks or probably don't PC networks on the backside. So when they
start looking at production, they have firewalls in the way or DMZ's and
they start thinking about how do I secure this. Then they get into the
point of well I'm not going to invest in new solutions I'll just ping-pong
out of the network interfaces and go through my physical hardware.
Latency, performance, it's just an absolute nightmare right now. VMware is
doing a good job with VMCI, get's us further along down the field in terms
of making some of the realities of not having to deploy these software
extentiations, that by the way are usually built on dedicated just to get
a gig put through a box. VM ware right now has done a Lenux to Lenix test
on a four core machine, I think it's four core, it shows 2.5 gigabits per
second, through put rates at large packet sizes between VM's on the same V
switch. And that's just one VM to another VM. If you tried to force 5,
10, of 20 VM's there, all doing 2.5 gigabits of transfer through put.
You're telling me that one little virtual appliance that's trying to vet
for the same set of resources on that box is going to be able to that? You
can't make it 2.5 gigs in a hardware FPJ box. So we've got lot's of fun
stuff to look forward to then.
Dino Dai Zovi: I think also we have to recognize too that one of the
problems with a lot of this stuff is the adoption of it is... I think we
like to think sometimes that people are doing these top down assessments of
what their environments going to be like. And it's not like that. I had
family member who will remain nameless because I would be permanently
disfigured if I mentioned her. But in her particular environment the
application owners don't actually touch the box; they have to manage
everything remotely. Whoever does the database has is somebody else on the
outside, and then server guys, when they use the server and the network is
a whole different group, and in the work stations, it's a different group.
This is just a midsized organization, not a big organization. They ran out
of power capacity in their data center. It's hard to call it a data
Chris Hoff: A data club?
Dino Dai Zovi: Yeah. They ran out of power capacity and they ran out of air
conditioning. And they have no possible way that they can do that with a
Capital reconstruction flaw. So the word comes out, virtualize whatever
you can. So all of the sudden you've got the server admin just clumping
the stuff into boxes. They don't understand the performance implications.
They're not buying new hardware. They're talking existing hardware, from
VM's on there, and thinking you have the CPU overhead of each virtual
machine, then you have to make sure that you have the right amount of
memory for each one of those and then you throw the V switch in. It's not
like virtualization all of the sudden makes all of your processors run
faster. All of that stuff is still using the same set of resources. Even
a multicore processor, if you don't configure that and throw those things on
a different shared board. You saw shared memory among those things I mean
it's just a freaking mess from a technical level, if you start clumping
Rich Mogull: So we need to start thinking about virtualization as a
larger tool than people think of it as. I think a lot of people think of
virtualization as like the size of the tool is just installing the
hypervisor. We'll just install the free VM server on it. And all of the
sudden we're virtualized. We can do this, and this, and this our problems
will be solved. I think you should think about at least a decent work
flow, a decent sized infrastructure with a hand full of servers running ESX
or something that bare metal hypervisor and then do something right.
Chris Hoff: You bring up a good point because hypervisors themselves
are becoming, and are a commodity. The reality is there's like 20 of them
already. Everyone's got one and the whole quest here is to make them
smaller and smaller, as possible. The reality to me is you look at
security there, security in that instance is a squeezing the balloon
problem. The physical size of the balloon doesn't change, you squeeze and
it pops out the other end. So what we're getting now, I make a point of
borrowing the line from "Lord of the Rings", 'There will be no ring 0 to
rule them all". The reality is hypervisors probably in a large
environment with five or six hypervisors, at least in the network.