Network booting with iPXE
1. ABSTRACT
Network booting is a feature, provided by nearly all widespread network interface cards. This feature allows a lot of advanced automation features to be deployed in various environments, which could decrease the amount of time that otherwise system administrators had to spend in order to complete tasks like massive server/client installations and/or re-installations, thin client solution deployments, system recovery of systems that do not have any other was of booting, like CD or USB and so on.
2. BACKGROUND
The founding stone of network booting is the PXE (Preboot eXecution Environment) environment, which is an environment to boot computers using a network interface independently of data storage devices (like hard disks) or installed operating systems [1]. This environment was originally introduced by Intel and uses several network protocols like Internet Protocol (IP), User Datagram Protocol (UDP), Dynamic Host Configuration Protocol (DHCP) and Trivial File Transfer Protocol (TFTP), and concepts like Globally Unique Identifier (GUID), Universally Unique Identifier (UUID) and Universal Network Device Interface (UNDI) [2].
Network booting process with PXE goes through the following steps, once BIOS handles the network booting to the network interface card firmware:
Step 1: PXE client sends extended DHCPDISCOVER request as a broadcast message in the network segment.
Step 2: A server that run DHCP service receives the extended DHCPDISCOVER request and reply to this request with an extended DHCPOFFER. The reply is sent to the IP address, which was originating the initial discover request, via unicast communication to port 68/UDP.
(There’s also a way to do this via proxy DHCP service, for environments where DHCP service cannot be deployed. For more information see PXE specifications link).
Step 3: After PXE client receives the extended DHCPOFFER from the DHCP server, it now can request its network configuration by sending an extended unicast DHCPREQUEST packet to the server that runs the DHCP service at 67/UDP.
Step 4: The DHCP server accepts the DHCPREQUEST packet and acknowledges it by sending unicast DHCPACK packet to the PXE client on port 68/UDP. With this packet the DHCP request procedure is complete and the PXE client is now configured with IP address and knows the boot server information, which was propagated as part of the extended DHCPOFFER. Boot server information includes the IP address of the server that runs TFTP service and the path to the network bootstrap program (NBP) on that TFTP server.
Step 5: The NBP is the program, which is executed by the PXE code after getting it from the TFTP server. This program is a boot loader, which allows us to boot the system from the network based on a configuration file. In the examples below we’ll use PXELINUX as our NBP.
Step 6: After PXELINUX took control from the PXE, it reads its configuration file, located at /tftpboot/pxelinux.cfg/default (/tftpboot/ is the root of out TFTP server in this example, but could be also anything else based on your server configuration; default is the final choice for configuration file, which PXELINUX tries. Before that it tries to load configuration files based on the client’s MAC address, then IP address (in HEX) and finally it tries default. If all of this fails it halts with an error and network booting has to be started from the beginning) and boots the default configuration in this file. For more information check: http://syslinux.zytor.com/wiki/index.php/PXELINUX
Step 7: The PXELINUX handles the control to the kernel that was configured and from there on the system continues to be managed by it.
All of the above said was pretty much explaining the network booting process from the standard PXE point of view. The goal of this article is to explain how to use iPXE instead, which gives us a wide range of capabilities for performing the network booting rather than just limiting us to booting from TFTP server.
iPXE, as described by its creators stands for ‘it doesn’t PXE’ and is a fork of the Etherboot Project, which is maintaining the gPXE. This fork is pretty much continuation of the gPXE, because the fork was done by the main developers of the gPXE and literary since the fork the contribution to gPXE project is virtually equal to zero [3].
iPXE could be either burned to a ROM of a network interface card and then executed directly from it by the BIOS, same way as PXE is invoked, or it could be chainloaded using the already available at the network card PXE code. In below examples, we’ll show how to perform network booting from a HTTP server by using the chainloading process, because if we have to perform a network booting of huge amount of servers and we do not want to use PXE this is the most convenient way.
For further reading on the topic the following links could provide you with sufficient information:
ftp://download.intel.com/design/archives/wfm/downloads/pxespec.pdf
http://pxe.dev.aboveaverageurl.com/index.php/PXE_Booting
http://ipxe.org/docs
3. ENVIRONMENT
In below couple of bullets I’ve described the lab environment that was used for building this scenario, plus I’ve included a simple diagram of the device layout and .
3.1. DIAGRAM
3.2. HARDWARE
1x Server device
1x Client device
3.3. SOFTWARE
CentOS Linux
DHCPD (ISC dhcpd used in the example)
XINETD
TFTPD (tftp-hpa used in the example)
HTTPD (Apache httpd used in the example)
3.4. CONFIGURATION
Below are included only the configuration files, which were modified. The default configuration files are not included.
/etc/dhcpd.conf:
## General options ############################################################ allow booting; allow bootp; option domain-name "higaknowit.com"; ddns-update-style none; next-server 10.1.1.1; ## iPXE specific options ############################################################ option space ipxe; option ipxe-encap-opts code 175 = encapsulate ipxe; option ipxe.priority code 1 = signed integer 8; option ipxe.keep-san code 8 = unsigned integer 8; option ipxe.skip-san-boot code 9 = unsigned integer 8; option ipxe.no-pxedhcp code 176 = unsigned integer 8; option ipxe.bus-id code 177 = string; option ipxe.bios-drive code 189 = unsigned integer 8; option ipxe.username code 190 = string; option ipxe.password code 191 = string; option ipxe.reverse-username code 192 = string; option ipxe.reverse-password code 193 = string; option ipxe.version code 235 = string; option iscsi-initiator-iqn code 203 = string; ## iPXE feature indicators ############################################################ option ipxe.pxeext code 16 = unsigned integer 8; option ipxe.iscsi code 17 = unsigned integer 8; option ipxe.aoe code 18 = unsigned integer 8; option ipxe.http code 19 = unsigned integer 8; option ipxe.https code 20 = unsigned integer 8; option ipxe.tftp code 21 = unsigned integer 8; option ipxe.ftp code 22 = unsigned integer 8; option ipxe.dns code 23 = unsigned integer 8; option ipxe.bzimage code 24 = unsigned integer 8; option ipxe.multiboot code 25 = unsigned integer 8; option ipxe.slam code 26 = unsigned integer 8; option ipxe.srp code 27 = unsigned integer 8; option ipxe.nbi code 32 = unsigned integer 8; option ipxe.pxe code 33 = unsigned integer 8; option ipxe.elf code 34 = unsigned integer 8; option ipxe.comboot code 35 = unsigned integer 8; option ipxe.efi code 36 = unsigned integer 8; option ipxe.fcoe code 37 = unsigned integer 8; ## If no ProxyDHCP servers are being used ## disable iPXE waiting for replies from ProxyDHCP ############################################################ option ipxe.no-pxedhcp 1; ## Enable PXE chainloading ############################################################ if exists user-class and option user-class = "iPXE" { filename "http://10.1.1.1/boot.php"; } else { filename "undionly.kpxe"; } ## DHCP subnet ############################################################ subnet 10.1.1.0 netmask 255.255.255.0 { range dynamic-bootp 10.1.1.10 10.1.1.254; default-lease-time 600; } |
/etc/xinetd.d/tftp
service tftp { disable = no socket_type = dgram protocol = udp wait = yes user = root server = /usr/sbin/in.tftpd server_args = -v -v -v -s /tftpboot -m /tftpboot/tftp-rules per_source = 11 cps = 100 2 flags = IPv4 } |
/tftpboot/pxelinux.cfg/default
DEFAULT menu.c32 MENU TITLE Installer PROMPT 0 TIMEOUT 150 MENU WIDTH 80 MENU MARGIN 16 MENU ROWS 15 MENU TABMSGROW 20 MENU CMDLINEROW 20 MENU TIMEOUTROW 21 MENU HELPMSGROW 22 LABEL CLIENT MENU DEFAULT MENU LABEL Install CLIENT DEVICE KERNEL vmlinuz INITRD initrd.img APPEND devfs=nomount ramdisk_size=10000 |
4. SOLUTION EXPLANATION
Below will follow a brief explanation of what the above mentioned software and its configuration are accomplishing.
Step1: First thing to do is to start the server and make sure that the dhcpd daemon is running and the configuration provided above is in use.
Example: This could be checked by running the following commands:
[root@server ~]# ps faux | grep dhcpd | grep -v "pts" |
The expected output from above command will look like:
[root@server ~]# ps ax | grep dhcpd | grep -v "pts" 32203 ? Ss 0:02 /usr/sbin/dhcpd eth4 [root@server ~]# |
In order to make sure that the right configuration file is used, check whether /etc/dhcpd.conf is the same like the one above and then run:
[root@server ~]# service dhcpd restart Shutting down dhcpd: [ OK ] Starting dhcpd: [ OK ] [root@server ~]# |
This will ensure us that the daemon has been started with the right configuration.
Step 2: As a second step we have to check whether the XINETD daemon is running and the above mentioned configuration is in place.
Example:
[root@server ~]# ps ax | grep xinetd | grep -v "pts" 6731 ? Ss 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid [root@server ~]# |
Check whether /etc/xinet.d/tftp is the same like the one above and then run:
[root@server ~]# service xinetd restart Stopping xinetd: [ OK ] Starting xinetd: [ OK ] [root@server ~]# |
Step 3: As a third step we have to check whether HTTPD is running. In this example we could use the default configuration of the HTTPD daemon.
Step 4: We have to check whether undionly.kpxe, boot.php and client.ks are in /var/www/html/ folder.
Step 5: Once we performed all of the above checks we could start the client machine and choose F12 for network booting. This will invoke the PXE environment from the client's NIC firmware, which will go through the above described PXE process.
The difference from above is that once the NIC's PXE client sends the DHCPREQUEST packet to the DHCP server it'll be pointed from the later to the undionly.kpxe first (on the TFTP server). This is the chainloading mechanism, which allows us to use iPXE, even though it is not burned in the NIC ROM.
The chainloading process is based on the following: The DHCP server checks the user class option from the client's packet and determines, whether the request is coming from iPXE client or standard PXE client. If request is comping from the PXE one DHCP server points it to the iPXE executable on the TFTP server, which is then loaded by the client. If the request is coming from iPXE client - it receives the URL of the iPXE script, which is parsed then by the iPXE.
The content of the script looks like this:
#!ipxe dhcp net0 set 209:string pxelinux.cfg/default set 210:string http://10.1.1.1/ chain ${210:string}pxelinux.0 |
The above instructs the following:
1. Refresh IP address from the DHCP server
2. Set the path to PXELINUX configuration file
3. Set the root of the PXELINUX container
4. Load PXELINUX from the container, which we setup in the previous step
Once we reach this point PXELINUX is loaded and it displays the menu based on its configuration file. Above is the example file. Once the timeout defined in the example file expires, or a selection is made, the specified CentOS kernel is going to be loaded from the HTTP server.
From here on the kernel takes control.
5. Reference List
1. ^ Preboot Execution Environment [Online] - http://en.wikipedia.org/wiki/Preboot_Execution_Environment
2. ^ Preboot Execution Environment [Online] - http://en.wikipedia.org/wiki/Preboot_Execution_Environment
3. ^ iPXE Frequently Asked Questions [Online] - http://ipxe.org/faq