TCP/IP Security


Department of Computer and Information Science, Ohio State University, Columbus, Ohio 43210

The TCP/IP protocols, the basis for today's Internet, lack even the most basic mechanisms for security, such as authentication or encryption. As usage of the Internet and TCP/IP protocols increases, their lack of built-in security has become more and more problematic. This paper describes a variety of basic flaws in TCP/IP protocols and their implementations, and discusses solutions and work-arounds to these problems. Also covered is the new IPv6, the next-generation Internet protocol that, among other goals, seeks to fix many of the current flaws in the current Internet IPv4 protocol. Security in protocols and applications not essential to TCP/IP (such as HTTP, FTP, and SMTP) are not discussed in this paper.


In the early 1980's, specifications were finished for the TCP and IP protocols. These two protocols could be considered the most important in the world today - they are the basis of the Internet. Over the past decade, the Internet has grown from a small network connecting a small community of researchers to its present state - a gigantic global network connecting people of all types. The huge success of the Internet has, for the most part, been quite beneficial. The Internet has evolved from a specialized project to a general-purpose tool. However, the growth of the Internet has created problems with security. The TCP and IP protocols were designed when the Internet was small, and users generally trusted each other. The protocols lack many features that are desirable or needed on an insecure network. In this paper, we present a number of problems that arise with using TCP/IP on today's network, solutions or work-arounds for these problems, and how the protocols for tomorrow's Internet, such as IPv6, will help eliminate these problems.

1. Introduction to TCP/IP

TCP/IP is the backbone of the internet today. Comprised of two protocols, TCP and IP, the TCP/IP protocol suite is one of the most widely used. We present a brief introduction to the two protocols. For a detailed discussion of the two protocols we refer the reader to the RFC's for IP [RFC 791], and TCP [RFC 793].

1.1. Internet Protocol

The Internet Protocol (or IP as it generally known), is the network layer of the Internet. IP provides a connection-less service. The job of IP is to route and send a packet to the packet's destination. IP provides no guarantee whatsoever, for the packets it tries to deliver. The IP packets are usually termed datagrams. The datagrams go through a series of routers before they reach the destination. At each node that the datagram passes through, the node determines the next hop for the datagram and routes it to the next hop. Since the network is dynamic, it is possible that two datagrams from the same source take different paths to make it to the destination. Since the network has variable delays, it is not guaranteed that the datagrams will be received in sequence. IP only tries for a best-effort delivery. It does not take care of lost packets; this is left to the higher layer protocols. There is no state maintained between two datagrams; in other words, IP is connection-less.
Type of Service
Total Length
Fragmentation Offset
Time to Live
Header Checksum
Source Address
Destination Address

Figure 1. IP Header

The IP Header is shown in Figure 1. The Version is currently set to 4. In order to distinguish it from the new version IPv6, IP is also referred to as IPv4. The source address and the destination address are 4-byte Internet addresses. The Options field contains various options such as source based routing, and record route. The source based routing allows the sender to specify the path the datagram should take to reach the destination. Record route allows the sender to record the route the datagram is taking. None of the IP fields are encrypted and there no authentication. It would be extremely easy to set an arbitrary destination address (or the source address), and IP would send the datagram. The destination has no way of ascertaining the fact that the datagram actually originated from an IP address other than the one in the source address field. It is easy to see why any authentication scheme based on IP-addresses would fail.

1.2. Transmission Control Protocol

Transmission Control Protocol (TCP) runs on top of IP, and provides a connection oriented service between the sender and the receiver. TCP provides guaranteed delivery, and ensures that the packets are delivered in sequence. The underlying network IP, is highly unreliable and does not provide any guarantee for TCP. In order to provide reliability between the sender and the receiver, TCP uses various mechanisms, such as sequence numbers, acknowledgments, 3-way handshakes and timers.

A TCP connection is identified by the 4-tuple ((destination-ip-address, destination-port), (source-ip-address, source-port)). Ports are the actual end-points of the TCP connection. The working of TCP could be described using a TCP state machine. (see Figure 5). Transitions to different states are based on events received in the form of TCP segments. The TCP states are very closely associated with timers. There are various timers associated with connection establishment (or termination), flow control, and retransmission.

In order to understand the security problems associated with TCP, it is necessary that we look at the state-machine in detail. It is also important to get an overview of TCP implementations, and how they implement the TCP state-machine, the state-transitions and the associated timers.

The TCP layer on either end maintains table entries corresponding to the 4-tuple (remote-ip-address, remote-port, source-ip-address, source-port). This 4-tuple uniquely identifies a connection. For every connection, the end-systems implementing TCP need to keep the TCP state information for the duration of the connection.

1.3. TCP Sequence Numbers

TCP is run over an unreliable IP, and IP does not guarantee delivery of packets, and does not necessarily deliver the packets in sequence. Delayed packets showing up is not uncommon. Given these conditions, TCP does not make any assumptions about the underlying network. In order to identify packets and ensure that they are delivered to the application layer in order, the sequence number is an important component of every TCP segment. Every byte of data that TCP sends is given a sequence number. By doing this, the sender and receiver can verify whether the data was delivered correctly. They can also determine whether data was dropped, possibly because of loss in transit.

Both the sender and the receiver exchange initial sequence numbers (ISN) during the connection setup phase. After a successful initial handshake, both the sender and the receiver know the sequence numbers that they have to use for communication. Since TCP allows for delayed segments, it must accept segments that are out of sequence, but within certain bounds, known as the receiver window size. The receiver window size is also exchanged during the initial handshake. TCP will discard all segments that do not have a sequence number within the computed bounds.

The TCP sequence number is a 32-bit counter. In order to distinguish between different connections between the same sender and receiver, it is important that the sequence numbers do not start at 0 or any other fixed number each time a connection is opened. Hence, it is important that the first byte of data from the sender to the receiver is using a random sequence number. Current implementations increment the sequence number by a finite amount every second.

1.4.TCP Header

Every TCP segment must contain a TCP header. The header format is the same for all TCP segments ( Figure 2). The header contains the source and destination port. The sequence number identifies the first byte of data following the header. The acknowledgment number indicates an acknowledgment up to (acknowledgment - 1) bytes by the receiver. There are six flag bits in the header, namely URG, ACK, PSH, RST, SYN and FIN. One or more of the flags can be set in any segment header.
16-bit source port number
16-bit destination number
32-bit sequence number
32-bit acknowledgment number
header length and flags
16-bit window size
16-bit TCP checksum
16-bit urgent pointer
Options (if any)
Data (if any)

Figure 2. TCP header format

The window size indicates the receiver window size, used for flow control. The TCP header and the data are covered by the checksum. There are other optional fields, not relevant to our discussion.

1.5. The Three Way Handshake

Most network protocols follow the classical three way handshake to establish or terminate connections over a not-so-reliable link. A initiates a connection to B by sending a message. B responds with an acknowledgment. At this point, A sends another message back to B confirming that A received B's acknowledgment. A and B connect successfully when B receives the second message from A (confirming B's ack). The three way handshake is used by TCP both to establish connections as well as to terminate the connections. When hosts A and B want to communicate using TCP, they need to establish a connection using the three-way handshake. Termination or closing a connection is also executed using the three-way handshake.

The hosts on either end identify their TCP connection with the other using the (IP-address, port number) combination. A table entry is created on receiving the first segment from the remote host which has all the parameters to identify the connection.

1.6. Connection Setup

TCP uses the 3-way-handshake to set up a successful connection. When host A wants to open a connection to host B, A sends an initial segment to B. This initial segment has the Initial Sequence Number (ISN) that B needs to use to send data to A. This initial segment is identified by the SYN bit set to 1 in the TCP header. If the SYN bit is set, the 32-bit sequence number in the header is interpreted as the ISN. In all other cases (when the SYN bit is not set), the 32-bit sequence number identifies the sequence number of the first data byte contained in that segment. B on receiving the SYN from A, has to respond with another SYN, as well acknowledge the SYN sent by A. This is indicated by SYN+ACK in the state machine diagram.

Figure 3. Initial Connection establishment.

1.7. Connection release

Connection release in TCP also uses the 3 way handshake. Connection release uses the FIN in place of the SYN.

Figure 4. Connection release in TCP

1.8. TCP Timers

Timers are closely knit with the TCP states. Some values are specified by the RFC, but not all. Some timers are left open to individual implementations. We will look at timers relevant to our discussion, which include the timers at connection establishment, and at connection termination.

1.9. Problems with the TCP State machine

The TCP statemachine is shown in Figure 5. There is a statemachine for every connection. Each connection logically starts in the CLOSED state, and makes transitions as shown in the diagram. After the connection is terminated, TCP returns to the CLOSED state. For a detailed description of the statemachine, refer [RFC 793]. It is easy to exploit a few flaws in the state-machines, and create denial-of-service attacks. All the denial-of-service attacks created try to stall the TCP statemachine in a particular state either indefinitely or for a finite time.

Figure 5. TCP State Machine
(from TCP/IP Illustrated, Vol. 1 by W. Richard Stevens)

1.10. Absence of timers

Looking at the state machine, we can see that there are no timers associated with certain states. As noted earlier, TCP does not send any data on the connection, unless used with a special option, the Keep-alive timer. This means that TCP state machine for a connection could be made to stay at such states forever. Most implementations do not implement timers for those states. An example of such a state is the CLOSE_WAIT state. If the keep-alive timer is used, TCP would be able to reset the connection. The default value of the keep-alive timer is set to 2 hours, which means that the state-machine is frozen atleast for a period of 2 hours.

1.11. Simultaneous connection establishment

When two hosts, say A and B want to establish a connection and both of them simultaneously initiate the handshake, we have a case of simultaneous connection establishment [RFC 793]. Both hosts A and B send out SYN's to each other. When the SYN's are received by the corresponding peers, both of them send out a SYN+ACK. Both hosts A and B must detect that the SYN and SYN+ACK actually refer to the same connection. If both hosts A and B detect that the SYN+ACK belongs to the SYN that was recently sent, they switch off the connection establishment timer and move directly to the SYN_RECVD state. This flaw could be used to stall a port on a host, using protocols such as FTP where the server initiates a connection to the client.

As an example, consider host X which has started a FTP connection to a server A. X and A are connected using the control-port. A initiates the connection establishment procedure to initiate data transfer with X.

Thus, X here is able to create a denial-of-service attack using this flaw in TCP.

1.12. SYN+FIN

The TCP specification does not specify clearly certain transitions and hence allows for some spurious state transitions. [Guha, Mukherjee]. These transitions could be used for a variety of attacks, especially the denial-of-service attacks. As an example, we will look at a scenario where a host receives a TCP segment with both the SYN and the FIN bit set.

On receiving a packet with the SYN and the FIN bit set, TCP makes a transition to the CLOSE_WAIT state. There has been no successful connection establishment and hence TCP should not be making such a transition. Most implementations do this because the specifications have failed to address the issue. The transition is clearly a bad one, since CLOSE_WAIT is a state with no timer associated. The receiver gets stalled in the CLOSE_WAIT state.

2. Problems in the TCP/IP protocol suite

When TCP/IP was designed in the early 1980's, security was not a primary concern. However, in the years since their inception, the lack of security in the TCP/IP protocols has become more of a problem. The widespread use and availability of the TCP/IP protocol suite has exposed its weaknesses. Presented here are a number of well-known vulnerabilities of both TCP/IP itself, and of some protocols commonly used along with TCP/IP (such as DNS).

2.1. TCP "SYN" attacks

In an Internet environment, high message latency and loss are not uncommon, resulting in messages that arrive late or in nonsequential order. The TCP half of TCP/IP uses sequence numbers so that it can ensure data is given to the user in the correct order, regardless of when the data is actually received. These sequence numbers are initially established during the opening phase of a TCP connection, in the three-way handshake.

SYN attacks (also known as SYN Flooding) take advantage of a flaw in how most hosts implement this three-way handshake [Guha, Mukherjee95]. When Host B receives the SYN request from A, it must keep track of the partially opened connection in a "listen queue" for at least 75 seconds. This is to allow successful connections even with long network delays. The problem with doing this is that many implementations can only keep track of a very limited number of connections (most track only 5 connections by default, though some like SGI's IRIX track up to 1024 [Luckenbach96]). A malicious host can exploit the small size of the listen queue by sending multiple SYN requests to a host, but never replying to the SYN&ACK the other host sends back. By doing so, the other host's listen queue is quickly filled up, and it will stop accepting new connections, until a partially opened connection in the queue is completed or times out. This ability to effectively remove a host from the network for at least 75 seconds can be used solely as a denial-of-service attack, or it can be used as a tool to implement other attacks, like IP Spoofing.

Figure 6. SYN Flooding

2.2. IP Spoofing

IP Spoofing is an attack where an attacker pretends to be sending data from an IP address other than its own [Morris85, Bellovin89]. The IP layer assumes that the source address on any IP packet it receives is the same IP address as the system that actually sent the packet -- it does no authentication. Many higher level protocols and applications also make this assumption, so it seems that anyone able to forge the source address of an IP packet (called "spoofing" an address) could get unauthorized privileges.

However, there are two catches. The first catch is that all communication is likely to be one-way. The remote host will send all replies to the spoofed source address -- not to the host actually doing the spoofing. So, an attacker using IP spoofing is unlikely to see output from the remote system (unless they have some other method of eavesdropping on the network between the other two hosts). The second catch is that an attacker needs to use the correct TCP sequence numbers if they plan on establishing a TCP connection with the attacked host (most common services, like Telnet, FTP, and r-commands use TCP). The final ACK in a three-way handshake must contain the other host's ISN, otherwise the connection cannot complete; because the ISN in the SYN+ACK packet is sent to the real host, an attacker must get this ISN by some other method. If the attacker could eavesdrop on the packets send from the other host, he could see the ISN. Similarly, if the attacker was unable to eavesdrop, but could somehow guess the other host's ISN, he can complete the connection and conduct a one way conversation (this may be sufficient to initiate some other form of two-way communication). Unfortunately for the TCP/IP community, methods to overcome both challenges in IP Spoofing have been developed.

2.2.1. Sequence Guessing

The sequence number used in TCP connections is a 32 bit number, so it would seem that the odds of guessing the correct ISN are exceedingly low. However, if the ISN for a connection is assigned in a predictable way, it becomes relatively easy to guess. This flaw in TCP/IP implementations was recognized as far back as 1985, when Robert Morris described how to exploit predictable ISN's in BSD 4.2, a Unix derivative [Morris85]. In BSD 4.2, the ISN for a connection is assigned from a global counter. This counter is incremented by 128 each second, and by 64 after each new connection (i.e., whenever an ISN is assigned). By first establishing a real connection to the victim, the attacker can determine the current state of the system's counter. The attacker then knows that the next ISN to be assigned by the victim is quite likely to be the predetermined ISN, plus 64. The attacker has an even higher chance of correctly guessing the ISN if he sends a number of spoofed IP frames, each with a different, but likely, ISN.

However, when the host receiving spoofed packets completes its part of the three-way handshake, it will send a SYN&ACK to the spoofed host. This host will reject the SYN&ACK, because it never started a connection -- the host indicates this by sending a reset command (RST), and the attacker's connection will be aborted. To avoid this, the attacker can use the aforementioned SYN attack to swamp the host it is imitating. The SYN&ACK sent by the attacked host will then be ignored, along with any other packets sent while the host is flooded. The attacker then has free reign to finish with his attack. Of course, if the impersonated host happens to be off-line (or was somehow forced off-line), the attacker need not worry about what the victim is sending out.

Figure 7. IP Spoofing via Sequence Guessing

Incrementing the ISN counter more often, as other operating systems may do, does not appear to help. Even if the counter is incremented 250,000 times a second (as suggested by the TCP standard), an attacker may still be able to approximately predict the ISN. By repeatedly guessing, it is likely that the attacker will establish a connection with the correct ISN within a few hours [Bellovin89].

2.2.2. Source Routing

Another variant of IP spoofing makes use of a rarely used IP option, "Source Routing" [Bellovin89]. Source routing allows the originating host to specify the path (route) that the receiver should use to reply to it. An attacker may take advantage of this by specifying a route that by-passes the real host, and instead directs replies to a path it can monitor (e.g., to itself or a local subnet). Although simple, this attack may not be as successful now, as routers are commonly configured to drop packets with source routing enabled.

Figure 8. Source Routing

2.3. Connecting Hijacking

An interesting variant on IP spoofing allows a host to insert itself in the middle of a connection between two hosts -- connection hijacking [Joncheray95]. IP spoofing alone may not bypass additional security, such as authentication by the Unix password mechanism, Kerberos, or one-time password systems like SKEY [RFC1760]. But with this attack, an attacker can allow normal authentication to proceed between the two hosts, and then seize control of the connection.

Connection hijacking exploits a "desynchronized state" in TCP communication. When the sequence number in a received packet is not the same as the expected sequence number, the connection is said to be "desynchronized." Depending on the actual value of the received sequence number, the TCP layer may either discard or buffer the packet. There is a choice, because TCP uses a sliding window protocol to allow efficient communication even in the presence of packet loss and high network latency. So, if the received packet is not the one expected, but is within the current window, the packet will be saved on the premise that it will be expected later (various TCP mechanisms ensure that the expected packet will eventually arrive). If the received packet is outside of the current window, it will be discarded.

Thus, when two hosts are desynchronized enough, they will discard (ignore) packets from each other. An attacker can then inject forged packets with the correct sequence numbers (and potentially modify or add commands to the communication). Obviously, this requires the attacker to be located on the communication path between the two hosts so that he may eavesdrop, in order to replicate packets being sent. The key to this attack is creating the desynchronized state. Joncheray describes two possible ways to do this: one is during the three-way handshake, and the other is in the middle of an established connection.

Note that "ignored" packets may actually generate ACKs, rather than being completely ignored. When the other end receives packets with incorrect sequence numbers, it replies with an ACK packet containing the sequence number it is expecting. But the receiver of these ACK discards them, as they have the wrong sequence numbers! The receiver then sends its own ACK to notify the sender... Thus, a large number of ACKs are generated in this attack. This "signature" of the attack could be used to detect connection hijacking.

Figure 9. Connection Hijacking

2.3.1. Desynchronization during connection establishment

In this form of desynchronization, the attacker resets a connection during the three-way handshake. After host B sends the SYN&ACK packet to host A, the attacker forges new packets from B (to A) in which the connection is first closed via the RST bit, and then a new three-way handshake is initiated with A -- identical to the original, "real" handshake but with different sequence numbers. Host B now ignores messages from A (because A is using the attacker's new sequence numbers), and Host A ignores messages from B (because A is expecting messages with the attacker's sequence numbers).

The attacker then replicates new packets, with the correct sequence numbers, whenever A and B try to communicate. In doing so, the attacker may also modify the messages or inject his own.

2.3.2. Desynchronization in the middle of a connection

The previous attack is limited to the initial connection. If a RST packet is sent in the middle of a connection, the connection is closed -- and the application/user is notified of this. To cause desynchronization in the middle of a connection, without closing the connection, only the sequence number counters should be altered. The Telnet protocol, in particular, provides an interesting mechanism to do this. Telnet allows special "NOP" commands to be sent. These commands do nothing, but the act of sending the bytes in the NOP command increments the expected sequence number counter on the receiver. By sending enough of these NOP commands, an attacker can cause the connection to become desynchronized. The attacker can then begin replicating new packets, with the correct sequence numbers, as before.

2.4. Routing (RIP) attacks

Although it is not strictly a component of TCP/IP, the Routing Information Protocol ("RIP") is often an essential component in a TCP/IP network [RFC1058]. RIP is used to distribute routing information within networks, such as shortest-paths, and advertising routes out from the local network. Like TCP/IP, RIP has no built in authentication, and the information provided in a RIP packet is often used without verifying it. Attacks on RIP [Bellovin89] are different from those of other common attacks because RIP attacks change where data goes to, not where it came from. For example, an attacker could forge a RIP packet, claiming his host "X" has the fastest path out of the network. All packets sent out from that network would then be routed through X, where they could be modified or examined. An attacker could also use RIP to effectively impersonate any host, by causing all traffic sent to that host to be sent to the attacker's machine instead.

2.5. ICMP attacks

The Internet Control Message Protocol ("ICMP") is used by the IP layer to send one-way informational messages to a host. One of the most common (and well-known) uses of ICMP is the "ping" utility. This utility sends an ICMP "Echo Request" to a host, and waits for that host to send back an ICMP "Echo Reply" message. Other messages in ICMP are of similar complexity; that is, they are all quite simple. It's not surprising that there is no authentication in ICMP, which leads to attacks using ICMP that can result in a denial of service, or allowing the attacker to intercept packets [Bellovin89].

Denial of service attacks primarily use either the ICMP "Time exceeded" or "Destination unreachable" messages. The "Time exceeded" message indicates that the Time-To-Live field in the IP header has expired; this can normally be caused by routing loops or trying to reach a host that is extremely distant. "Destination unreachable" messages can have several meanings (based on a sub-field in the ICMP message), but all basically indicate that packets cannot successfully be sent to the desired host. Both of these ICMP messages can cause a host to immediately drop a connection (this is the desired result if the ICMP message is legitimate). An attacker can make use of this by simply forging one of these ICMP messages, and sending it to one or both of the communicating hosts. Their connection will then be broken.

ICMP messages can also be used to intercept packets. The ICMP "Redirect" message is commonly used by gateways when a host has mistakenly assumed the destination is not on the local network (and is thus attempting to send the packet via the gateway to). If an attacker forges an ICMP "Redirect" message, it can cause another host to send packets for certain connections through the attacker's host. This attack is similar to a RIP attack, except that ICMP messages only apply to existing connections, and the attacker (the host receiving redirected packets) must be on a local network.

2.6. DNS attacks

The Domain Name Service ("DNS") is a protocol widely used on the Internet. DNS is primarily used to map hostnames (i.e. "") to IP addresses (i.e., but it can also be used to do the reverse; mapping IP addresses to hostnames. An attacker can use the latter property to fool name-based authentication [Schuba, Spafford94; Bellovin89]. For example, an administrator at may decide to allow only local connections. This is often specified by name, such as "allow *," rather than by IP address. Name-based authentication is easier to read, and allows easier administration if a domain contains multiple ranges of IP addresses. When a connection is established with, alice uses DNS to convert the source IP address on the connection to a name, which is then checked using whatever form of name-based authentication the administrators have installed. If an attacker has access to their local DNS server, they can cause DNS queries on their IP address to reply with any hostname and domain! So, an attacker who knows that trusts connections from within * can alter his DNS server so that his IP address appeared to map to "" Today, this attack is often prevented by performing a second DNS query on the hostname returned by the first query. If the IP address returned by the second query does not match the source IP address from the original connection, the host knows that a DNS attack is likely being attempted. Proposals to modify DNS to include mechanisms to ensure data integrity and authentication may also mitigate the vulnerabilities in DNS [RFC2065]

2.7. The lack of unique identifiers

When the Internet was still young, back in the 1980's, one could use an IP address to safely identify and locate a host. Addresses were both spatially unique (no one else had an identical address) and temporally unique (addresses didn't change). However, this is no longer the case [RFC2101]. Today, IP addresses may exhibit seemly strange behavior that makes identifying and locating hosts much harder. The widespread use of protocols such as PPP/SLIP [RFC1990] and DHCP [RFC1541] allow a specific host's address to change over time: per-connection in the case of PPP/SLIP, while DHCP allows hosts to "lease" IP addresses for arbitrary lengths of time. On even larger time scales, details in the current Internet routing structure (i.e., Classless InterDomain Routing, "CIDR" [RFC1519]) may require that if a domain changes service providers, they will have to change their assigned range of IP addresses. Firewalls, proxy socket servers, and other "Network Address Translators" further complicate the use of IP addresses as identifiers, because they may translate addresses as traffic moves between the internal and external networks. Different hosts may appear to be using identical IP addresses, or different IP addresses may be the same host. Thus, IP addresses can no longer be used to uniquely identify a host, even over short time periods. Any security schemes which rely upon IP addresses remaining temporally or spatially unique may have vulnerabilities.

3. Solutions to Problems

There are a plethora of papers discussing theoretical weaknesses in the TCP/IP protocol, with more being published all the time. For each paper which warns readers about "Security Weakness X," there seems to be a corresponding chunk of code that some individual has written which utilizes this weakness in real life, on actual computers. Corresponding to the SYN flooding attack there is a SYNflood.c freely available for download, which, when compiled and run, will attempt to deny all access to a target host by flooding it with connection requests. There are packages (spoofit is one) which offer menu-driven interfaces for connection-hijacking. Indeed, there are actual, working implementations of all security weaknesses discussed so far. Going from the paper to the "exploit", as the implementations are known, is sometimes not a trivial task; it often requires significant hands-on networking knowledge. The user of the exploit, on the other hand, is not required to understand all the details; the exploits are often well-documented and user-friendly. With more people downloading them every day, it is unremarkable that we have seen an increase in break-ins over the last few years [Bellovin92].

Given that there are often well-written, easy to use implementations of the security vulnerabilities discussed so far, it should come as no surprise that much effort has gone into the elimination of these vulnerabilities. Since TCP/IP is the medium linking the entire Internet together, it is easy to see that billions of dollars worth of hardware are at risk here, discounting entirely the monetary worth of the information on those machines. It would be very nice to squelch these vulnerabilities. Unfortunately, most of the attacks described in this paper are based on inherent vulnerabilities in the TCP/IP protocol; without reworking TCP/IP, there may be little that can be done! While this may indeed be what eventually solves these problems, it is not a task to be undertaken lightly; if there is any way to avoid having to rebuild the Internet with a new version of TCP/IP, people will prefer it. Let's explore what can be done within the current incarnation of TCP/IP.

3.1. Problems with Accepting Connections

The SYN attacks described earlier are used as a denial-of-service attack to prevent anyone from using a particular host. Often times, this is used as a stepping stone towards a more complicated spoofing attack. This first stepping stone is one of the inherent vulnerabilities in TCP/IP, based on what it was designed to do.

As a connection-oriented protocol used to link together machines across the world, TCP/IP must accept some variance in connection-times; i.e. the computers in New York expect to make connections with other computers in New York as well as computers across the globe. Since the time for messages to pass back and forth between the computers may vary, it is reasonable to allow a generous amount of time for the connection to be completed. As long as this amount of time is finite, then a nearby computer will be able to send a large number of uncompleted connection-requests, and eventually tie up the target host. The very fact that we want TCP/IP to be friendly and connect to computers of varying distances means that these SYN attacks are difficult to eradicate completely.

There are, however, some choices that can be made to decrease vulnerability. Increasing the number of allowable outstanding connections can make more work for the attacker (or dynamically allocating them instead of relying on a fixed maximum). Another practical solution is to decrease the timeout period (currently 75 seconds) to something smaller, thereby increasing the workload for the attacker again. In practice, these solutions may reduce the vulnerability to something negligible, but it should be noted that so long as the timeout period and maximum number of incomplete connections is finite, the problem still exists.

3.2. Problems with Authentication

The utter lack of authentication with IP packets is a general weakness with TCP/IP.

Without authentication, there really is no guarantee that a packet comes from where its source field claims it comes from. This is the foundation of all IP spoofing and consequently, the major issue in IP security. There are many different ways to capitalize on this weakness; we have provided an overview of the problems that come about from the lack of authentication, but our discussion of the myriad ways this may be taken advantage of could never really be complete.

Since a lack of authentication is the broadest, more crippling weakness in TCP/IP, we would really like to clamp down on it as much as possible. Since the conventional IP version 4 does not typically use authentication, it is not possible to completely eradicate the threat in the most ideal way, by providing it at the IP level. However, we cannot simply throw up our hands in defeat; there are several techniques we may employ to blunt this weapon.

3.2.1. Preventing Sequence Guessing

For host A to successfully impersonate host B in a connection with host C, A must be able to acquire the sequence number which C sends back (to B). There are a number of ways to get at this number, but one popular technique is to simply guess the number that will be used. If we could prevent people from being able to guess the next sequence number used for a connection based on the previous sequence number used in a connection, we would be able to nip this problem in the bud.

In Bellovin's paper, Security Problems in the TCP/IP Protocol Suite, he outlines an essential problem with the TCP specifications, at least from a security standpoint. Berkeley systems are extremely easy to analyze; their method of incrementing the sequence counter by a constant value every second, and incrementing it by half that for every connection established is trivial to anticipate. But the Berkeley system does not follow TCP specifications, which require the variable to be incremented approximately 250,000 times per second. Perhaps if we followed the specifications for the protocol we could eliminate sequence guessing.

"Let us consider whether a counter that operated at a true 250,000 hz rate would help. For simplicity's sake, we will ignore the problem of other connections occurring, and only consider the fixed rate of change of this counter. To learn a current sequence number, one must send a SYN packet, and receive a response. The first spoof packet, which triggers generation of the next sequence number, can immediately follow the server's response to the probe packet. The sequence number used in the response is uniquely determined by the time between the origination of the message and the receipt at the server of the message. But this number is precisely the round-trip time between [spoofer] and [server]. Thus, if the spoofer can accurately measure (and predict) that time, even a 4 microsecond clock will not defeat this attack.

How accurately can the trip time be measured? If we assume that stability is

good, we can probably bound it within 10 milliseconds or so. Clearly, the Internet does not exhibit such stability over the long-term, but it is often good enough over the short term. There is thus an uncertainty of 2500 in the possible value for [the guessed sequence number]. If each trial takes 5 seconds, to allow time to re-measure the round-trip time, and intruder would have a reasonable likelihood of succeeding in 7500 seconds, and a near certainty within a day. More predictable (i.e., higher quality) networks, or more accurate measurements, would improve the odds even further in the intruder's favor. Clearly, simply following the letter of the TCP specification is not good enough." [Bellovin89]

So it appears that even following the TCP specifications will not work. What can be done? Bellovin goes on to suggest that we utilize a random number generator, which would make things hopefully harder to analyze. He further suggests that DES could be used in place of a random number generator, since it is well-studied, and the encryptions of an incrementing counter are quite varied and difficult to predict. Of course, all methods of generating pseudorandom numbers are subject to analysis; if nothing else, once the seed is discovered, the whole sequence is known. Whatever we choose as a solution, we should certainly avoid using a simple, easily predicted sequence number generator.

3.2.2. The Benefits of Firewalls

A firewall can be a powerful tool in the prevention of would-be spoofers. Putting aside the proxy-services normally offered by firewalls, we concentrate on the benefits derived from packet filtering techniques. The important part about firewalls from an IP spoofing perspective is that they clearly delineate outside the firewall from inside the firewall; everything inside must go through the 'inside' port on the firewall, and everything outside must come in through the 'outside' port [Ranum92]. This means that the packet filtering done in the firewall can drop suspicious packets! Suppose the filter sees a packet come from the outside that claims to have a source inside the firewall. It's a spoofed packet, and should be dropped; it's claiming to come from inside, but it's coming from outside [Chapman92]. Likewise, if some packet attempts to leave the firewall claiming to be from anywhere other than inside the known subnet, it can be dropped immediately as well [Ranum92]. In a sense, this sort of filtering partitions the Internet into little zones, none of which can spoof each other. However, even with this sort of filtering going on, spoofing within the subnet cannot be prevented.

3.2.3. The Benefits of TCP Wrappers

Additional protection against authentication weaknesses can be gained from the use of TCP wrappers. A TCP wrapper is a small program run in place of the services which the inet daemon normally starts. For example, with TCP wrappers in place, when a user attempts to rlogin, the standard rlogin program is not run directly; first, the tiny wrapper program is run, which performs some security functions, and then, assuming everything checks out, the actual rlogin program is run. These are a trivial thing to install, nothing like the implementation of a firewall.

"A typical implementation provides tiny daemon wrapper programs that can be installed without any changes to existing software or to existing configuration files. The wrappers report the name of the client host and of the requested service; the wrappers do not exchange information with the client or server applications, so are essentially invisible to the users." [Venema92]

The usual benefit gained from installing a wrapper is that you can log information about

which users are requesting which services, and track suspicious behavior, or suspected persons. Venema discusses how it is possible to add tripwires to the TCP wrapper packages; instead of merely dropping users who are attempting to use some service when they shouldn't, one can reverse-finger them to gain additional tracking information. Or, one can turn on a more stringent security standard, expecting an attack. And there are some added perks which help defeat some forms of spoofing: some of the inet services, notably rsh and rlogin, rely on name-based authentication, which can be spoofed [Schuba and Spafford94]. If a local copy of names and addresses is available, a TCP wrapper can verify that someone starting up rsh or rlogin has an IP address that agrees with the name they claim. Also, it is likely that a machine may not have local tables, and might have to ask DNS for a name-IP correspondence... this leaves the machine very vulnerable to anyone doing DNS spoofing. A TCP wrapper can be configured to ask multiple DNS servers for the correspondence and wait for consensus.

The benefits that can be gained from TCP wrappers are not in any way impregnable armor; the consensus is that they are not useful in preventing the general case of IP spoofing. However, they do provide some armor against specific variations on the spoofing problem, and they provide a space for extra checks to be inserted where they may not be expected. For a further discussion of this topic, see [Venema92].

3.2.4. Add-On Authentication: Kerberos

Another method of limiting an attacker's spoofing abilities is to add authentication onto the application layer. Of course, just adding authentication is not enough without adding encryption; otherwise, after some initial application-level authentication, a hijacking attack may still be successful. An authentication between two parties which exchanges a session key, however is secure; even though the IP packets transmitted back and forth are not individually authenticated, they are all encrypted with the secure session key. This scheme is the goal of the Kerberos Authentication System, developed at MIT. The Kerberos system uses cryptographic authentication algorithms to ensure that a user is really who s/he claims to be, and once this is established, an exchanged session key is used to encrypt all transmissions of whatever service the user has requested. Without knowledge of this session key, it is impossible for an attacker to spoof meaningful transmissions between sources. Since this key is generated based on secret keys known only to the actual user and the trusted server, it is very hard for an attacker to acquire. The Kerberos system is resilient to replay attacks as well. Kerberos is generally considered to significantly increase the security of a network, although it is not a panacea. There are problems using Kerberos to authenticate between two machines (instead of a user and a machine), and there are difficulties involving where the keys are cached on a multiuser machine. For a discussion of weaknesses in Kerberos, see [Bellovin, Merritt91] .

3.2.5. Encrypting Individual IP Packets (SKIP)

Instead of exchanging a session key, as we might do via Kerberos for a telnet session, we could choose to encrypt all IP packets, all the time, at the IP level. Naturally, we must encrypt them in a way that the destination can successfully decrypt them. There is a special key-distribution scheme designed for packet-level encryption called the Simple Key-Management for Internet Protocols (SKIP). SKIP assumes that each site in the network has a public key, which can be used to create many keys between two sites. The basic idea of the algorithm is to encapsulate the packet-key (the key to decrypt that packet) inside the packet, and encrypt that with shared secret between the two sites. The SKIP technique also provides an easy method of changing the shared secret between two sites [Aziz, Patterson96]. Since this method is not session-based, it can cover all aspects of TCP/IP communication, not merely applications.

4. IP Version 6

The problem of authentication within IPv4 is a challenging one. While firewalls, wrappers, Kerberos and SKIP may help curb spoofers, they are add-ons designed to shore up the deficiencies in IPv4. They are not guarantees of security, and they involve significant efforts for each network that wants to implement them. Looking back, it would have been better if security had been a goal of the TCP/IP protocol suite from the beginning.

Although it is not possible to go back in time and change history, we can look forward to the future. In the 1990s, TCP/IP security is a concern, and a newer version of the protocol has been developed with this concern in mind. In fact, it has been developed with a number of new concerns in mind. One important motivating factor for the development of a new protocol is that the address space is running out. A lot of experience has been gained in the years of usage of IPv4 about what tasks IP should perform, and the new version of IP is redesigned to reflect that experience. The new version will eventually replace the current version. The current version of the protocol is 4, and the new version designed is version 6, referred to as IPv6. IPv6 attempts to address the newfound knowledge of the importance of routing, security, etc. It promises to provide authentication and encryption on the Internet and could solve a lot of the existing problems with TCP/IP. For a complete description of IPv6, we refer the reader to [Huitema]. We will restrict our focus to the security features introduced in IPv6.

IPv6 Header

The IPv6 header consists of 64-bit initial header fields, followed by 128-bit source addresses and destination addresses.
Version Priority
Flow Label
Payload Length Next HeaderHop limit
Source Address
Destination Address

1Figure 10. IPv6 Base Header

The header shown in Figure 10 is the IPv6 base header. There are no options in the base-header. Instead, the options are specified as extension headers. These extension headers are placed between the base header and the payload. Payload here refers to the data that is carried by IPv6. The current IPv6 specification defines six extension headers:

For example, if there was an authentication header following the IPv6 base header, we will have a chain of headers as shown in Figure 11. Each header indicates the type of the next header that immediately follows it. In Figure 11, the Base Header indicates that the next header that follows itself is the Authentication header. The Authentication Header indicates that the TCP header and data follow.

Figure 11. Chaining of IPv6 Header

Security options in IPv6

IPv6 includes two extension headers that serve as security options, the Authentication Header and the Encryption header. The Authentication Header allows the recipient to ascertain the identity of the sender and the encryption Header ensures that only the recipient is able to look at the contents of the message. Both these options in IPv6 require that the sender and the receiver agree on parameters such as the key, the authentication or the encryption algorithm, and the lifetime of the key. This is termed as a security association in IPv6. The recipient of a packet has to verify or decrypt only in the context of the security association between the sender and the receiver. The security association or the security context is identified using the security parameter index (SPI). The SPI is normally negotiated as part of the key-exchange procedure.

The SPI is usually chosen by the receiver for every (sender, receiver) pair. The receiver uses the SPI to identify the specific security context. In case of multi-casting, where there are multiple receivers, the SPI will be common to all the members of the multicast group. The SPI will be used by all the members to correlate the security context.

Key Distribution in IPv6

The establishment of security associations is crucial to both authentication and encryption. During a security association, the keys, other information about the keys such as the life-time of the key, the authentication and the encryption algorithms are exchanged between a receiver and a sender. In addition to this, this association context can be later referred to using the SPI, the Security Parameter Index. Efficient deployment of IPv6 security will rely on an efficient key distribution mechanism. It is desirable that the key distribution be a zero-knowledge exchange, that is the sender and the receiver should be able to exchange keys without agreeing on any secret a-priori.

The Internet community has not yet agreed upon a key distribution method. The leading proposal is called Photuris, which is a modification of the Diffie-Hellman key-exchange algorithm. [Diffie, Hellman]

Diffie Hellman Algorithm

Diffie-Hellman was the first public-key algorithm invented, and dates back to 1976. It can be used to generate a secret key, but not for encrypting and decrypting messages.

Assume parties A and B are involved and want to generate a secret key. A and B agree on a large prime number, n and a generator g, such that g is primitive mod n. The two integers, n and g do not have to be secret. A and B can agree to them over an insecure channel.

  1. A chooses a large random number x, and computes
    X = gx mod n
    A then sends X to B.
  2. B chooses a large random, y and computes
    Y = gy mod n
    B sends Y to A.
  3. A computes
    k = Yx mod n
  4. B computes
    k' = Xy mod n

After the initial exchange between A and B, A knows x and Y, and B knows X and y. Since this is exchanged over an insecure link, X and Y could be found out by a third-party and are public. But either x or y is not obtainable by the third party. Steps 3 and 4 yield the session key Z,
Z = k = k' = gxy mod n

The computed key Z, can then be used by either the encryption or the authentication algorithm.
The Diffie-Hellman algorithm derives its security from the difficulty of calculating discrete logarithms in a finite field, as compared with the ease of exponentiation in the same field. It is difficult for a third-party to compute the discrete logarithm and recover x or y. The choice of g and n is important, and n must be large. As observed earlier, g must be primitive mod n. One could choose g as small as possible, and usually it is a one-digit number. The security of the system is based on the difficulty of factoring numbers the same size of n. Other mathematical techniques, such as elliptic curve groups could also be used for the key exchange.


The Diffie-Hellman algorithm has two distinct advantages, that could be made use of in protocols.


The original Diffie-Hellman algorithm has a known set of weaknesses.


Photuris tries to rectify certain problems in the Diffie-Hellman algorithm. The algorithm was named Photuris as a tribute to unknown engineers. The Internet draft, by Phil Karn and Simpson describes Photuris. [Karn96]. It will probably be revised before being published as an Internet RFC.


Photuris starts using a cookie exchange, which precedes the Diffie-Hellman key exchange. After the keys are exchanged, signed messages are sent over the new protected channels.

All Photuris messages are exchanged using UDP, the unreliable transport equivalent of TCP.

The Photuris key exchange

  1. A sends a cookie-request to B.
    Cookie-request ( initiator cookie, group list)
    The cookie request contains a 128-bit initiator cookie. The initiator cookie is a random number used as an identification for the key-exchange. The cookie request also contains a list of groups. These groups designate the finite fields over which the Diffie-Hellman exchange is carried out. The current draft supports three groups: the elliptic curve group, the exponential group based on a 1024-bit prime with 2 as the generator, and another exponential group with 5 as the generator.
  2. B on receipt of the cookie-request, selects one of the groups provided by A and constructs a cookie-response to send to A.
    Cookie-response( initiator cookie, responder cookie, selected group, responder attributes)
    The cookie-response from B contains the responder-cookie. The responder-cookie is computed by applying a one-way function such as MD5 to a string composed of the source address and the UDP port of the message, the initiator cookie and a secret number chosen by B. B does not usually maintain any memory of the exchange. B also sends a list of attributes to A. The list of attributes contains a set of transforms in order of B's preference. This feature allows A and B to use any of the listed methods for signatures and encryption. The standard specifies MD5 as the default one-way function to be used for Authentication, and DES-CBC as the encryption scheme.
  3. A and B now enter the Diffie-Hellman exchange phase, where they exchange the half-keys and generate a session key k. The Key-request message is sent by A to B. The message contains A's security parameter index (SPI), in addition to the Diffie-Hellman public value. A lifetime for the request is also sent in the key-request message. The initiator transform field indicates the transforms chosen by A out of B's transform priorities. A also sends its own transform priority list using the Initiator attributes. The Group Index, indicates the mathematical group to be used in the Diffie-Hellman scheme. A indicates its value for the Security Parameter Index in the initiator SPI field.
    Key Request (Group Index, Initiator Lifetime, initiator SPI, Initiator Cookie, Responder Cookie, Initiator Public Value, Initiator Transform, Initiator Attributes)
  4. B sends its Diffie-Hellman half-key using the Key-response message. The parameters to the message are similar to the Key-request message.
    Key Response ( Responder Lifetime, Responder SPI, Initiator Cookie, Responder Cookie, Responder Transform, Responder Public Value, Key Transform)
  5. The cookie exchange and the Diffie-Hellman exchange do not protect against the man-in-the-middle attack. This problem can be solved by requiring A to send its signature to B, and B could verify the signature. This assumes that A has B's certificate, and B has A's certificate. These certificates might have been signed by a trusted authority outside the protocol.
    (initiator-cookie, responder-cookie, signature, certificate) : A B and B A
    A signs the session-key concatenated with the public value of the key, B's public value, and optionally the identifying public key, certificate, or the distinguishing name of A. Inclusion of both public values in the signature guarantees that a third party, the middle man has not substituted its own value. If B is able to verify A's signature correctly, then B sends its own signature to A, for A to authenticate B. In case of a failure, the security association is dropped.

Key Distribution for Multicast groups

The Diffie-Hellman scheme can be easily extended for three or more parties. [Schneier96]. But for multi-casting, it is ideal to have a key-server distribute a single key to all the parties concerned.

The Authentication Header in IPv6

IPv6 uses the daisy chaining of options to support authentication as well. One of the extension header that can be sent after the base header could be an authentication header (AH). The AH has a very simple format as shown in Figure 12. The SPI or the Security Parameters Index, denotes the security context of the sender and the receiver. SPI would be used by the receiver to retrieve the security context of the sender.

The sender computes a signature of the payload data, some fields of the IPv6 header and the extension headers, and this signature is sent as the authentication data. The algorithm used for the signature is negotiated as part of the key-exchange process. The MD5 algorithm is specified as a default to make sure all implementations can at least use one common algorithm. The receiver retrieves the appropriate security context, using the SPI indicated and verifies the signature.

There are intrinsic details in creating the signature that are interesting to note. The Authentication header has to ensure that the entire IPv6 packet is not modified or tampered with. But, there are certain fields in the IPv6 header that need to be modified by routers on the way. For example, the hop count field is decremented at every hop. Some hop by hop options may also be updated in transit. In order to take care of this, the sender needs to prepare a special version of the message that is independent of the changes that can take place in transit, such as setting the hop count to zero.

The Authentication Option could be used between routers or between end-nodes. Using it between routers could solve common routing attacks that are prevalent today. The IP spoofing attacks could be prevented by using the Authentication option.
Next Header
Security Parameters Index (SPI)
Authentication Data

Figure 12. IPv6 Authentication Header

Encryption Header in IPv6

The Encrypted Security Payload (ESP) header, when used forms the last of the extension headers in the chain of headers (see Figure 13). All data following the encryption header is encrypted, and the data preceding the encryption header is in plain-text. The format of the encryption header is shown in Figure 14. The SPI is used by the receiver to retrieve the security context of the sender, and then decrypt the data following it.

IPv6 Base Header

Extension Headers

ESP header

Encryption header

Figure 13. Encryption using the ESP header

The encryption scheme is negotiated during the key-exchange. DES-CBC is the default encryption scheme that is supported.
SPI (Security Parameter Index)
Encrypted Data And Parameters

Figure 14. The ESP Header

Application of Security Options in IPv6

IPv6 has introduced network layer authentication and encryption. This enables the use of both encryption and authentication by all protocol layers above IPv6. The routing protocols, that are totally insecure today, could be made secure using IPv6 security options. Secure channels could be established over the insecure Internet.

Securing Routing protocols

Network integrity depends on the integrity of the routing protocols. If the routing protocols are not secure, then they cannot guarantee network integrity. The protocols that are used today between routers, RIP, OSPF and others are run over an insecure link. If intruders forge the routing updates, then they will be able to disrupt communication or divert connections. IPv6 security options could be used to secure the routing protocols.

Securing Neighbor discovery

Neighbor discovery is an important characteristic of IPv6. This feature allows auto-configuration, also known as, plug and play. A new host should be able to configure its address automatically when it is plugged into a network. However, having this feature also introduces new security holes. A host that is not authorized to the network may use the services. Auto-configuration depends on neighbor discovery, an algorithm to enable a host to find out its neighbors. A host may get updates from illegal nodes, and builds wrong routing tables.

There is a chicken-and-egg problem associated with neighbor advertisements. In order to secure the neighbor advertisements, a security association is required between the two parties. But, the neighbor advertisements are also the first packets exchanged between the two neighbors. The two parties cannot exchange keys and do the neighbor advertisements as well. The problem could be solved by a deferred security association. Another solution would be to change the routing protocols such that the first next hop is always fixed, which could be a router. The router could forward the first packet to the required destination. Both the sender and receiver could later update their local tables using authenticated redirects from the router.

Securing tunnels between firewalls

At present, most organizations secure their intranet using firewalls. There is still a problem when two geographically different divisions of the same organization that are connected over the Internet. Both divisions have secure firewalls, but each division will be insecure when they have to communicate over the Internet while communicating to the other. A secure channel between the two firewalls is required. Using the IPv6 options, a secure channel could easily be established between the two firewalls.

Problems with IPv6 Security Options

The inclusion of the Authentication and the Encryption options the network layer is certainly welcome, and proposes to solve a lot of the current TCP/IP security problems. The design of IPv6 has considered the existing problems that prevail on the Internet. There are certain problems, however, in implementing security in any network-layer protocol today. The question of choosing between link-level encryption and application-level encryption has always plagued the researchers. The network is growing faster, and using encryption is going to slow down the protocols, thereby not utilizing the entire bandwidth available. Thus, the Internet Engineering Task Force has yet to decide on a Key Exchange protocol.

5.0 Summary and Conclusions

TCP/IP, as it exists today, has a general lack of security. Examples of implementations of SYN flooding, IP Spoofing, Connection Hijacking, etc. show that this lack of security has lead directly to the development of tools and techniques to exploit TCP/IP's weaknesses. Fixing some of these flaws today is possible (with add-ons like TCP Wrappers, Kerberos, and SKIP), but these fixes are not always in widespread, everyday use - your host may implement them, but host you want to communicate with may not. Thus, most communication on today's Internet is still unsecured. Migrating to a new protocol suite, such as IPv6, may be the only answer to fixing the flaws in TCP/IP for good.