You get this when the address is already in use. (Oh, you figured that much out?) The most common reason for this is that you have stopped your server, and then re-started it right away. The sockets that were used by the first incarnation of the server are still active. This is further explained in 2.7 Please explain the TIME_WAIT state., and 2.5 How do I properly close a socket?.
When you issue the close() system call, you are closing your
interface to
the socket, not the socket itself. It is up to the kernel to close the
socket. Sometimes, for really technical reasons, the socket is kept
alive for a few minutes after you close it. It is normal, for example
for the socket to go into a TIME_WAIT state, on the server side, for a
few minutes. People have reported ranges from 20 seconds to 4 minutes
to me. The official standard says that it should be 4 minutes. On my
Linux system it is about 2 minutes. This is explained in great detail in
2.7 Please explain the TIME_WAIT state..
There are two approaches you can take here. The first is to use inetd to do all the hard work for you. The second is to do all the hard work yourself.
If you use inetd, you simply use stdin, stdout, or stderr for
your
socket. (These three are all created with dup() from the real socket)
You can use these as you would a socket in your code. The inetd process
will even close the socket for you when you are done.
If you wish to write your own server, there is a detailed explanation in "Unix Network Programming" by Richard Stevens (see 1.5 Where can I get source code for the book [book title]?). I also picked up this posting from comp.unix.programmer, by Nikhil Nair ( nn201@cus.cam.ac.uk):
I worked all this lot out from the GNU C Library Manual (on-line
documentation). Here's some code I wrote - you can adapt it as necessary:
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <unistd.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/wait.h>
/* Global variables */
...
volatile sig_atomic_t keep_going = 1; /* controls program termination */
/* Function prototypes: */
...
void termination_handler (int signum); /* clean up before termination */
int
main (void)
{
...
if (chdir (HOME_DIR)) /* change to directory containing data
files */
{
fprintf (stderr, "`%s': ", HOME_DIR);
perror (NULL);
exit (1);
}
/* Become a daemon: */
switch (fork ())
{
case -1: /* can't fork */
perror ("fork()");
exit (3);
case 0: /* child, process becomes a daemon: */
close (STDIN_FILENO);
close (STDOUT_FILENO);
close (STDERR_FILENO);
if (setsid () == -1) /* request a new session (job control) */
{
exit (4);
}
break;
default: /* parent returns to calling process: */
return 0;
}
/* Establish signal handler to clean up before termination: */
if (signal (SIGTERM, termination_handler) == SIG_IGN)
signal (SIGTERM, SIG_IGN);
signal (SIGINT, SIG_IGN);
signal (SIGHUP, SIG_IGN);
/* Main program loop */
while (keep_going)
{
...
}
return 0;
}
void
termination_handler (int signum)
{
keep_going = 0;
signal (signum, termination_handler);
}
The best way to do this is with the select() call. This tells the
kernel
to let you know when a socket is available for use. You can have one
process do i/o with multiple sockets with this call. If you want to wait
for a connect on sockets 4, 6 and 10 you might execute the following code
snippet:
fd_set socklist;
FD_ZERO(&socklist); /* Always clear the structure first. */
FD_SET(4, &socklist);
FD_SET(6, &socklist);
FD_SET(10, &socklist);
if (select(11, NULL, &socklist, NULL, NULL) < 0)
perror("select");
The kernel will notify us as soon as a file descriptor which is less than
11 (the first parameter to select()), and is a member of our
socklist becomes
available for writing. See the man page on select() for more details.
This socket option tells the kernel that even if this port is busy, go ahead and reuse it anyway. It is useful if your server has been shut down, and then restarted right away while sockets are still active on its port. You should be aware that if any unexpected data comes in, it may confuse your server, but while this is possible, it is not likely.
It has been pointed out that "A socket is a 5 tuple (proto, local addr, local port, remote addr, remote port). SO_REUSEADDR just says that you can reuse local addresses. The 5 tuple still must be unique!" by Michael Hunter (mphunter@qnx.com). This is true, and this is why it is very unlikely that unexpected data will ever be seen by your server. The danger is that such a 5 tuple is still floating around on the net, and while it is bouncing around, a new connection from the same client, on the same system, happens to get the same remote port. This is explained by Richard Stevens in 2.7 Please explain the TIME_WAIT state..
On some unixes this does nothing. On others, it instructs the kernel to abort tcp connections instead of closing them properly. This can be dangerous. If you are not clear on this, see 2.7 Please explain the TIME_WAIT state..
From Andrew Gierth ( andrewg@microlise.co.uk):
The SO_KEEPALIVE option causes a packet (called a 'keepalive probe')
to be
sent to the remote system if a long time (by default, more than 2 hours)
passes with no other data being sent or received. This packet is designed to
provoke an ACK response from the peer. This enables detection of a peer
which has become unreachable (e.g. powered off or disconnected from the net).
See
2.8 Why does it take so long to detect that the peer died?
for further discussion.
Note that the figure of 2 hours comes from RFC1122, "Requirements for Internet Hosts". The precise value should be configurable, but I've often found this to be difficult. The only implementation I know of that allows the keepalive interval to be set per-connection is SVR4.2.
From Andrew Gierth ( andrewg@microlise.co.uk):
The restriction on access to ports < 1024 is part of a (fairly weak) security scheme particular to UNIX. The intention is that servers (for example rlogind, rshd) can check the port number of the client, and if it is < 1024, assume the request has been properly authorised at the client end.
The practical upshot of this, is that binding a port number < 1024 is reserved to processes having an effective UID == root.
This can, occasionally, itself present a security problem, e.g. when a
server process needs to bind a well-known port, but does not itself need
root access (news servers, for example). This is often solved by creating
a small program which simply binds the socket, then restores the real userid
and exec()s the real server. This program can then be made setuid root.
From Andrew Gierth ( andrewg@microlise.co.uk):
After accept()ing a connection, use getpeername() to get the
address of the client. To get the hostname, see
4.10 How do I use the gethostbyaddr() function?.
The client's address is of
course, also returned on the accept(), but it is essential to
initialise the address-length parameter before the accept call for this
will work.
From Andrew Gierth ( andrewg@microlise.co.uk):
Many people are confused by the fact that the address parameter to this
function is declared as char*. That doesn't mean it's a character
string representation of the address!
The first parameter should really have been declared as void*, not
char*;
but the functions probably precede this extension to the C language. If you
are using AF_INET addresses, then you should use a struct in_addr *, cast to a char*, as in the following example:
struct sockaddr_in addr;
struct hostent *host;
...
host = gethostbyaddr((char *) &addr.sin_addr, sizeof(addr.sin_addr),
AF_INET);
The list of registered port assignments can be found in STD 2 or RFC 1700.
Choose one that isn't already registered, and isn't in /etc/services on
your system. It is also a good idea to let users customize the port
number in case of conflicts with other un-registered port numbers in other
servers. The best way of doing this is hardcoding a service name, and
using getservbyname() to lookup the actual port number. This method
allows users to change the port your server binds to by simply editing
the /etc/services file.
SO_REUSEADDR allows your server to bind to an address which is in a
TIME_WAIT state. It does not allow more than one server to bind to the
same address. It was mentioned that use of this flag can create a
security risk because another server can bind to a the same port, by
binding to a specific address as opposed to INADDR_ANY. The
SO_REUSEPORT
flag allows multiple processes to bind to the same address provided all of
them use the SO_REUSEPORT option.
From Richard Stevens ( rstevens@noao.edu):
This is a newer flag that appeared in the 4.4BSD multicasting code
(although that code was from elsewhere, so I am not sure just who
invented the new SO_REUSEPORT flag).
What this flag lets you do is rebind a port that is already in use, but only if all users of the port specify the flag. I believe the intent is for multicasting apps, since if you're running the same app on a host, all need to bind the same port. But the flag may have other uses. For example the following is from a post in February:
From Stu Friedberg ( stuartf@sequent.com):
SO_REUSEPORTis also useful for eliminating the try-10-times-to-bind hack in ftpd's data connection setup routine. WithoutSO_REUSEPORT, only one ftpd thread can bind to TCP (lhost, lport,INADDR_ANY, 0) in preparation for connecting back to the client. Under conditions of heavy load, there are more threads colliding here than the try-10-times hack can accomodate. WithSO_REUSEPORT, things work nicely and the hack becomes unnecessary.
I have also heard that DEC OSF supports the flag. Also note that under
4.4BSD, if you are binding a multicast address, then SO_REUSEADDR is
condisered the same as SO_REUSEPORT (p. 731 of "TCP/IP Illustrated,
Volume 2"). I think under Solaris you just replace SO_REUSEPORT with
SO_REUSEADDR.
From a later Stevens posting, with minor editing:
Basically SO_REUSEPORT is a BSD'ism that arose when multicasting was added,
even thought it was not used in the original Steve Deering code. I
believe some BSD-derived systems may also include it (OSF, now Digital
Unix, perhaps?). SO_REUSEPORT lets you bind the same address *and* port,
but only if all the binders have specified it. But when binding a
multicast address (its main use), SO_REUSEADDR is considered identical
to SO_REUSEPORT (p. 731, "TCP/IP Illustrated, Volume 2").
So for portability of multicasting applications
I always use SO_REUSEADDR.
The original question was actually from Shankar Ramamoorthy ( shankar@viman.com):
I want to run a server on a multi-homed host. The host is part of two networks and has two ethernet cards. I want to run a server on this machine, binding to a pre-determined port number. I want clients on either subnet to be able to send broadcast packates to the port and have the server receive them.
And answered by Andrew Gierth ( andrewg@microlise.co.uk):
Your first question in this scenario is, do you need to know which subnet the packet came from? I'm not at all sure that this can be reliably determined in all cases.
If you don't really care, then all you need is one socket bound to
INADDR_ANY. That simplifies things greatly.
If you do care, then you have to bind multiple sockets. You are obviously attempting to do this in your code as posted, so I'll assume you do.
I was hoping that something like the following would work. Will it? This is on Sparcs running Solaris 2.4/2.5.
I don't have access to Solaris, but I'll comment based on my experience with other Unixes.
[Shankar's original code omitted]
What you are doing is attempting to bind all the current hosts unicast addresses as listed in hosts/NIS/DNS. This may or may not reflect reality, but much more importantly, neglects the broadcast addresses. It seems to be the case in the majority of implementations that a socket bound to a unicast address will not see incoming packets with broadcast addresses as their destinations.
The approach I've taken is to use SIOCGIFCONF to retrieve the list of
active network interfaces, and SIOCGIFFLAGS and SIOCGIFBRDADDR
to identify broadcastable interfaces and get the broadcast addresses.
Then I bind to each unicast address, each broadcast address, and to
INADDR_ANY as well. That last is necessary to catch packets
that are
on the wire with INADDR_BROADCAST in the destination.
(SO_REUSEADDR is
necessary to bind INADDR_ANY as well as the specific addresses.)
This gives me very nearly what I want. The wrinkles are:
This question is usually asked by people who are testing their server with telnet, and want it to process their keystrokes one character at a time. The correct technique is to use a psuedo terminal (pty). More on that in a minute.
According to Roger Espel Llima
(
espel@drakkar.ens.fr), you can have
your server send a sequence of control characters: 0xff 0xfb 0x01 0xff 0xfb 0x03 0xff 0xfd 0x0f3, which
translates to IAC WILL ECHO IAC WILL SUPPRESS-GO-AHEAD IAC DO
SUPPRESS-GO-AHEAD. For more information on what this
means, check out std8, std28 and std29. Roger also gave the
following tips:
0xff, it will be followed by two more characters.
These are telnet escapes.Use of a pty would also be the correct way to execute a child process and pass the i/o to a socket.
I'll add pty stuff to the list of example source I'd like to add to the faq. If someone has some source they'd like to contribute (without copyright) to the faq which demonstrates use of pty's, please email me!