12-4 IPsec VPNs

IPsec derives its name from the title of RFC 4301, that is, Security Architecture for the Internet Protocol. IPsec is a set of security protocols that work together to ensure security of IP traffic as it traverses the Internet.

IPsec can be used to secure IP traffic between:

• Two hosts
• Two security gateways (usually routers of firewalls)
• A host and a security gateway

IPsec not only provides encryption at the network/IP layer but also defines a new set of headers that are added to the encrypted IP packet. IPsec is flexible being a framework of open standards, and describes the messaging to secure communications, but relies on existing algorithms.

IPsec uses the concept of a security association (SA) to define a set of security parameters used for various VPN functions. SAs are used by AH and ESP as well as by the IKE protocol. SAs are created as a result of an IPsec VPN connection establishment between two hosts or two gateways. SAs are uni-directional in nature and there will be two SAs in place with each secure connection, one for each direction.

There are three main protocols in the IPsec framework. 

Internet Key Exchange (IKE)

Internet Key Exchange (IKE) is a combination of Internet Security Association and Key Management Protocol (ISAKMP), Oakley, and SKEME protocols. The names IKE and ISAKMP are sometimes used interchangeably in IPsec discussions though we prefer to use IKE in this chapter. IKE establishes authenticated keys and also negotiates security associations (SAs) that are then used by ESP and AH protocols. IKE uses UDP port 500.

IKE is a two-phase protocol: IKE phase 1 verifies the identity of the remote peer or in other words authenticates the remote peer. The two peers then establish an authenticated secure channel to communicate further. IKE offers two primary methods of authenticating a remote peer:

• Preshared Keys: It is the most common method that uses manually configured secret keys on both peers. It is easy to deploy but is not scalable and very secure.

• Public Key Signature: It uses the Public Key Infrastructure (PKI) and is the most secure method. 

At the end of phase 1 negotiation, an ISAKMP/IKE SA (phase 1 SA) is established. Phase 2 negotiations then take place over the secure channel established in phase 1.

IKE phase 2 negotiates SAs that are used to protect actual user data. At the end of phase 2 negotiations, two unidirectional IPsec SAs (phase 2 SAs) are established for user data. One SA is used for sending encrypted data and the other is used for receiving encrypted data. 

Authentication Header (AH)

Authentication Header (AH) provides data integrity and authentication for IP packets passed between two systems. It can be used when confidentiality is not required. It is used to verify that a message that has been passed from router A to router B has not been modified during transit. AH does not provide confidentiality and does not use encryption. All messages are sent in clear text, if the AH protocol is used alone which offers only weak security. However, AH is used in combination with other protocols like ESP to offer more robust security features. 

Encapsulating Security Payload (ESP)

Encapsulating Security Protocol (ESP) is a member of the IPsec protocol suite. It is an IP based protocol that uses IP port number 50 for communication between IPsec peers. It can provide authentication, integrity, confidentiality, and anti-replay protection of data. IP packet encryption not only hides the contents of the packet but also conceals the identities of the real source and destination found in the IP header in the form of source and destination IP addresses. ESP provides authentication for the encrypted IP packet and the ESP header. Authentication ensures data originated at a trusted source and was not modified during transit. 

IPsec Modes

IPsec has two modes of operation:

• Tunnel Mode: Tunnel mode secures data in site-to-site or network-to-network scenarios. In tunnel mode, the device performing VPN functions, such as a router or security appliance, does that on behalf of other users. In tunnel mode, the entire IP packet including the original IP header and the payload is encrypted and a new IP header is appended.

• Transport Mode: Transport mode secures data in host-to-host or end-to-end scenarios. In transport mode each user performs VPN functions on its own. In transport mode, IPsec protects the payload of the original IP packet but excludes the IP header. The transport mode, unlike the tunnel mode, preserves the original IP header and inserts the IPsec header between the original IP header and payload.

Both tunnel mode and transport mode can make use of ESP and AH protocols.

Cisco IOS defines bundles of encryption algorithms called transform sets that are used together to secure VPN traffic. IPsec transform sets define encapsulation (ESP or AH), encryption (3DEs or AES-128), authentication/integrity algorithm (MD5 or SHA-1), and the IPsec mode (transport or tunnel). You have the option to create your own custom transform sets though Cisco IOS also provides some defaults.

Table 12-2 Default IPsec Transform Sets in Cisco IOS