The blog of Karl Majer…

This article is one in a series detailing the implementation of a Multi-OS boot environment. While the article should stand on its own, it may be generally helpful to read the other articles in the series for the highest level of understanding.

The configuration of the PXE environment requires several components:

• dhcpd server
• tftpd server
• PXE bootloader

Each of these items has customized configuration files which will be described below.

At the lowest level there is a dhcpd server listening for initial boot requests. ISC’s dhcpd server, specifically version 3.0.6 which was the current version at the time. On the production RHEL machine, version 3.0.5 from the RPM was used.

The full intricacies of configuring the dhcpd file can be found in the manpage, however at a minimum one needs to specify the following pxelinux entries and create a single subnet stanza for basic functionality.

# pxelinux settings
option space pxelinux;
option pxelinux.magic code 208 = string;
option pxelinux.configfile code 209 = text;
option pxelinux.pathprefix code 210 = text;
option pxelinux.reboottime code 211 = unsigned integer 32;

#and

# Servers subnet
subnet 172.16.32.0 netmask 255.255.254.0 {
authoritative;
option routers 172.16.32.1;
option subnet-mask 255.255.254.0;
option broadcast-address 172.16.33.255;
option domain-name “dhcp.karlmajer.com karlmajer.com”;
range dynamic-bootp 172.16.33.201 172.16.33.251;

ddns-domainname “dhcp.karlmajer.com”;
ddns-rev-domainname “dhcp.33.30.172.in-addr.arpa.”;

next-server 172.16.32.16;

filename “boot/pxelinux.0″;
site-option-space “pxelinux”;
option pxelinux.magic f1:00:74:7e;
option pxelinux.configfile “pxelinux.cfg”;
option pxelinux.pathprefix “/tftpboot/pxelinux/files/”;
option pxelinux.reboottime 30;
}

Several assumptions are made by that configuration snippet that will need updating:

1. *There is no other dhcpd server on the network*
2. The tftpd server is at 172.16.32.16
3. The default router is at 172.16.32.1
4. You’ve placed your pxelinux and configuration files in $TFTPROOT/boot directory
5. Your network spans 2 class Cs, i.e. the 255.255.254 (/23) netmask and 33.255 broadcast.

After making these changes in the dhcpd.conf file, start up the daemon using the method of your choice, i.e., service dhcpd start, svcadm enable dhcpd, or even simply ‘dhcpd’ from the command line.

At this point there should be a functioning dhcpd server on the network. The next thing to configure is the tftpd server. Tftp-hpa version 0.42 was chosen as the tftp server of choice as it supported the remapping of file paths in requests, and offered tcp wrappers as an added bonus. Edit the tftpd.map file, often sought by tftpd at /etc/tftpd.map, and add the following lines to it:

rgi ^pxelinux.0 /boot/pxelinux.0
rgi ^pxelinux.cfg/ /boot/
rgi ^boot/pxelinux.cfg/ /boot/
rgi ^boot// /

This sets up the location of the boot files relative to the tftpd root which is specified on startup. In the environment that was built, all provisioning related items were placed in /export/install and the tftpd daemon was chrooted to this path. Therefore the pxelinux related files were in /export/install/boot, the menu files were in /export/install/hostmenu.

Now the tftpd server can be run from either inetd or as a standalone. Given the low usage of the install system, it was decided that tftpd would run from xinetd on RHEL ignoring the startup/shutdown costs of the daemon which may be noticeable were the install frequency several installs per hour rather than week.

Configuration of the inetd process will differ depending on the flavor of unix employed. On RHEL one need simply add a service definition to the xinetd.d directory and the xinetd daemon will pick it up on the next invocation. A sample RHEL stanza looks as follows:

# default: off
# description: The tftp server serves files using the trivial file transfer \
# protocol. The tftp protocol is often used to boot diskless \
# workstations, download configuration files to network-aware printers, \
# and to start the installation process for some operating systems. service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -m /etc/tftpd.map -v -v -v -s /export/install -t 10
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}

Note the explicit paths to the tftpd.map as well as the chroot to /export/install.

At this point we have a daemon to respond to our PXE requests, and we have a daemon to serve the content using the tftp protocol for the PXE requests. Now we need to supply the file to be served.

PXELinux from http://syslinux.zytor.com/pxe.php was used as the bootloader. The files were unbundled and placed into /export/install/boot. A menu file was created for pxelinux, the contents of which look like:

#Note, this menu starts the local disk, a memtest utility, and a Windows 2000 RIS #Installation.
default local
prompt 1
timeout 0
display /boot/install.msg
F1 /boot/install.msg

label local
localboot 0

label w50aa
kernel /images/W50AA/boot/w50aa.0

label memtest
kernel /boot/memtest
append -

For safety the default configuration chosen is to always favor the internal drives and only attempt to boot from the w50aa stanza on request. Now copy the menu containing the described data into the /export/install/boot/ (adjusted appropriately for your filesystem layout) and either name the file default, or name the file something else and symlink it to default.

At this point you should have the basic workings of a PXE loader environment. To test it first make sure that dhcpd is running:

ps -ef | grep dhcpd

Next use tftp to connect to localhost and request the file pxelinux:

tftp localhost
get pxelinux

Finally, reboot a host with a PXEboot capable network adapter and hit F12 to netboot.

Please let me know if you have any questions or comments.

>>> Karl

Continuing from where we left off in Part 1. In the /export/install/images/W51AA directory there are several directories: boot, i386, inf, amd64, and sys. The directories contain the following:

boot/ – the collection of bootloader files and the sif file.

inf/ – the driver information files. The contents of this dir is read by the python RIS implementation processes.

sys/ – the drivers themselves

i386/ – the cab files and other files found on the i386 directory on the media

amd64/ – on 64 bit OSes, this contains 64b versions of the i386 files.

The boot directory is populated as follows. Modifications need to be made to the actual windows binaries using your binary editor of choice. There are several strings, such as “winnt”, prevalent throughout all of the binaries and they each need to be replaced with a new string. This is done to create uniqueness across all of the windows install variants all of which request the same files such as winnt.sif, ntdetect.com, and whatnot. Using your binary or hex editor of choice, and substitute the following strings:

• startrom.n12 – NTDLR to w50aa
  • rename this file to w50aa.0
• setupldr.exe – winnt.sif to w50aa.sif; ntdetect to ntdw50aa;
  • rename this file to w50aa
• ntdetect.com – no string replacement
  • rename this file to ntw50aa.com

Alternatively, the following script snippet can be adopted to use. In the example below let ${Lwhich}=”w50aa”. The full script can be found at http://www.karlmajer.com/files/addnewos/addnewos.sh. Note, if using the script, the script assumes you’ve created the directory w50aa/i386 and/or w50aa/amd64 depending on the bitness of the OS and copied the files from the install media to those locations.

echo “Preparing boot directory.”
echo “…cabextracting files…”
cabextract -d boot/ i386/startrom.n1_
cabextract -d boot/ i386/setupldr.ex_

echo “…copying files…”
cp i386/ntdetect.com boot/

echo “…editing binaries…”
echo “…..startrom.n12…”

cat > /tmp/${Lwhich}.startrom.$$ << EOF
:%s/NTLDR/${Lwhich}/g
:wq!
EOF
vim -b -s /tmp/${Lwhich}.startrom.$$ boot/startrom.n12

echo “…..setupldr.exe…”

cat > /tmp/${Lwhich}.setupldr.$$ << EOF
:%s/winnt.sif/${Lwhich}.sif/g
:%s/ntdetect/ntd${Lwhich}/g
:wq!
EOF
vim -b -s /tmp/$Lwhich.setupldr.$$ boot/setupldr.exe

echo “…renaming binaries…”
mv boot/ntdetect.com boot/ntd${Lwhich}.com
mv boot/setupldr.exe boot/${Lwhich}
mv boot/startrom.n12 boot/${Lwhich}.0

You’ll note the vim is being used in binary mode (-b) in order to do the edits. The stubfiles simply create regex expressions for vim that are used to handle the search and replace. The script also handles the renames for you as well.

When it is all said and done, you will have the following files in the boot directory: w50aa, w50aa.0, and ntdw50aa.com. If you’re using the full version of the script then a w50aa.sif configuration file will have been created also. Those files map to the old ntldr, ntdetect, and then the setup specific files, setupldr.exe and the winnt.sif file.

To populate the i386 directory all that is required is to copy the i386 directory off the media directly into the tree.

For the inf and sys directories, the following needs to be done. Note that ${BIT} denotes 32 or 64 bit and pulls files from the appropriate directories accordingly. Again, this is included in the script linked above.

echo “Preparing inf and sys directories.”

if [ ${BIT} ];
then
echo “…extracting inf files…”
cabextract -d inf amd64/[Nn][Ee][Tt]*.[Ii][Nn]_
echo “…extracting sys files…”
cabextract -d sys -F *.sys amd64/driver.cab
else
echo “…extracting inf files…”
cabextract -d inf i386/[Nn][Ee][Tt]*.[Ii][Nn]_
echo “…extracting sys files…”
cabextract -d sys -F *.sys i386/driver.cab
fi

echo “Complete.”

That takes care of the directories. The only remaining things to do are to update the tftpd.map file and to create a few symlinks to fix bizarre windows naming conventions as well as update the windirs.conf which is the binl parser configuration file and update it to include this new OS we’ve added. Both of these are covered in detail in the script.

That concludes the preparation of the windows image. The next section will discuss setting up the low level boot and PXE environment.

>>> Karl

It takes a bit of work to prepare the contents of a windows CD image for use with the installation system. There is nothing terribly complicated about the process but it is a somewhat tedious, and after the 20th or so image that is being prepared, you may find yourself wishing for a script or two to simplify things. This is not to worry, I will provide those as well.

First a bit of background on the windows boot process. As a windows machine starts the boot process, the following things happen approximately in this order (1):

1. The MBR is queried for the boot sector, and Ntldr is read.
2. Ntldr is loaded into memory. Its reads the Boot.ini file to present the menu and loads the kernel.
   1. Ntldr will eventually load Ntoskernl.exe, Bootvid.dll Hal.dll and the boot-start device drivers.
3. Ntdetect.com then runs and performs a basic hardware detection for Ntldr.
4. Ntbootdd.sys is loaded for I/O.
5. Ntoskrnl.exe is initialized and starts the system-start device drivers then kicks off smss.exe
6. Hall.dll is loaded and is used to speak to the hardware in the machine.
7. Smss starts the windows subsystem.
8. Winlogon is started.
9. SCM (service control manager) is started for windows services/drivers.

The boot process across the network is somewhat similar:

1. The pxeclient on the network interface in computer drops a boot request onto the wire.
2. The dhcp server catches this request and gives the pxeclient an IP address, the IP address of the next server in the process, and the name of the file to fetch. .
   1. pxelinux was used for the windows installs.
3. The pxeclient then fetches pxlinux via tftp from the bootserver and presents the menu to the user and waits for entry.

Up until this point the pxe boot procedure is the same regardless of what OS is being loaded.

1. The user selects the desired windows installation and pxelinux tftps back to the server and asks for the appropriate file, typically startrom.n12, the network bootloader for the MS boot process.
2. startrom.n12 then calls for setupldr.exe
3. setupldr.exe, actually named ntldr on the filesystem, is the pre-installation setup loader for windows.
4. as before ntldr calls ntdetect.com for hardware detection,
5. similarly, ntdetect.com handles hardware detection for the ntldr process as for the disk based boot
6. finally the file winnt.sif is loaded which contains the instructions for the setup/RIS based installation.

Now, in order to make this all work a bit of filename manipulation magic needs to be undertaken. This is accomplished in the tftpd.map file which is loaded by the tftp server and additionally by making a few changes to both the names of the binaries and to the binaries themselves.

The naming convention chosen at the site was based on the fact that NTLDR is five characters. A method to encode OS names/types into the five characters was devised such that any OS/SP level could be encoded. The first letter was either W for 32 bit or X for 64 bit. The next two characters were the Microsoft OS version number, W50 for Windows 2000 32bit, X51 for Windows XP Pro 64 Bit. Finally the last two characters were used to denote patch level and any other desired information. For example, at the client site, X51AA was an unpatched XP Pro 64 Bit. While X51BA was SP2.

The path used on the filesystem to contain the filesystem images was /export/install/images. The tftp root is /export/install. The mapfile contained:

# windows to unix pathing
rgi \\ /

# anything asking for /IMAGES is broken and likely a Microsoft client.
rge ^/IMAGES/ /images/

# rehome 32 and 64 2k and 2k3 to /images
rgi ^/X5.* /images\0
rgi ^/W5.* /images\0
# Windows 2000 32 bit – W50AA

rgi ^/boot/pxelinux.0ntdw50aa.com /images/W50AA/boot/ntdw50aa.com
rgi ^boot/pxelinux.0ntdw50aa.com /images/W50AA/boot/ntdw50aa.com
rgi ^/boot/pxelinux.0w50aa.sif /images/W50AA/boot/w50aa.sif
rgi ^boot/pxelinux.0w50aa.sif /images/W50AA/boot/w50aa.sif
rgi ^w50aa /images/W50AA/boot/w50aa
rgi ^/images/W50AA/i386/W50AA/sys/(.*) /images/W50AA/sys/\1

# Windows XP Professional 64bit – X51AA
rgi ^x51aa /images/X51AA/boot/x51aa
rgi ^ntdx51aa.com /images/X51AA/boot/\0
rgi ^x51aa.sif /images/X51AA/boot/\0
rgi X51AA/i386 X51AA/amd64
rgi ^/images/X51AA/amd64/X51AA/sys/(.*) /images/X51AA/sys/\1
rgi ^/images/X51AA/i386/X51AA/sys/(.*) /images/X51AA/sys/\1

First off we need to rewrite dos paths to unix paths. Next we need to send anything asking for just /[WX]5 to /images. That allows the client to find the files its going to request next. The next section addresses some bugs in windows 2000 in which it tacks on the file its looking for onto the file it loaded previously, ie the pxelinux bootloader. Thankfully XP lacks this bug. You’ll notice that the filenames are not those mentioned in the process flow earlier, but are instead files based on the new naming convention. Any additional OSes based on 2000 need to simply copy the 2000 stanza and adjust the w50aa to the new 5 digit code. The same holds true for XP. These mappings allow the pxeclient to find the contents of the appropriate boot directory.

Configuring the directories in the W51AA tree will be covered in the next post.

>>> Karl

(1. This disk based bootup process was adapted from “Table 5-1, Microsoft Windows Internals, Fourth Edition, p252″. Any typos are mine. I’d recommend the original version of this from the text if possible. Its one of the few windows reference books on my shelf.)