If you need assistance, you may find it here. If you run into a strange problem that you can't figure out, feel free to email me as there may be a bug in flashrd that needs to be fixed.
root on rd0a swap on rd0b dump on rd0b fsck: /dev/wd0a: unknown special file or file system. fsck: /dev/wd0a: unknown special file or file system. bootstrap: fsck wd0a failed, attempting boot anyways mount_ffs: /dev/wd0a on /flash: Device not configured bootstrap: fatal error mounting wd0a to /flash Enter pathname of shell or RETURN for sh:
You didn't run cfgflashrd against your image (or disk.) The default image assumes your root device shows up as an older style ATA disk, wd0. That's true if you are using compact flash, bochs, or qemu. If you are using AHCI, USB key, or something else, your disk is probably sd0. You have to specify this using cfgflashrd -rdroot szez.blah.blahblahblah.
Say your USB key is sd1. Try 'fdisk sd1' and you will see:Disk: sd1 geometry: 974/255/63 [15663104 Sectors]cylinders is displayed (974), heads is also known as "tracks/cylinder" (255) and sectors is also "sectors/track" (63)
A USB key is a nice target for flashrd. It is cheap, easily removable and replacable without unracking and disassembling the entire machine. SLC usb keys are also cheap and recommended. Of course, flashrd images are easily upgradeable, as long as you have enough free space on the flash, so you should never need to remove the flash.
If you want to save things like dhcpd.leases on shutdown, you can simply set tardirs=var in /etc/rc.shutdown. If you want to do it manually (such as after you add or remove ports) then do: "tar cf /flash/var.tar -C /var ." In the case of dhcpd.leases, I suggest moving the dhcpd.leases file to the /flash partition (such as dhcpd -l /flash/dhcpd.leases) to ensure persistency.
Sure, just fire up the 'flashimg' under qemu. The i386 version usually works with the i386 qemu emulator. The amd64 version works, but at the time of writing this entry, the amd64 qemu emulator isn't so good. Bochs is another option. These tools are best for prototyping a real system.
flashrd has several advantages over non-ramdisk read-only schemes like flashdist and other variations of it. It also has advantages over other ramdisk based systems.
- Uses complete, default OpenBSD installation
- /etc/rc and everything following it runs completely unmodified, only the kernel, bootstrap, and partitioning are modified from default
- Running read-only system can be upgraded in-place by simply copying 3 files and rebooting (read-write systems can be upgraded by copying only 2 files and rebooting)
- Multiple versions of the system can exist on the same flash concurrently
- The bootstrap will attempt to run the older version, if the new version is not mountable
- No distribution list or specialized build system is required, flashrd packages up whatever binaries your system operates with
- flashrd can even create images from a running system, including any installed software
- Using a complete, default installation means that normal ports and packges can be installed and used on a flashrd system without hassle
flashrd boots from a ramdisk kernel with some basic utilities to mount a vnd image with all of your persistent partitions, and then untars the rest of the system on mfs. All partitions are defined in /etc/fstab, but are mounted by /stand/rc before the real /etc/rc is run. (The real /etc/rc is actually inside the vnd image, so this has to happen first! The partitions are remounted with the options in /etc/fstab by the real /etc/rc so you only have to change fstab to change how partitions are mounted.)The layout can be changed (including the size of any partition) by editing variables at the top of the 'flashrd' script.
tardirs are directories that are mounted on pre-sized ramdisks, thus they are not saved upon reboot (unless defined in etc/rc.shutdown tardirs variable). They must be defined in flashrd, etc/rc.conf.local and optionally in etc/rc.shutdown (if you want them saved on each shutdown) (by default, /var is the only member of tardirs)
mfsdirs are pre-sized ramdisks, with no content loaded to them. They must be defined in flashrd. (by default, /tmp)
vnddirs are saved to the openbsd.vnd image and mounted read-only. They must be defined in the same order in flashrd, cfgflashrd, stand/rc, bin/ro and bin/rw (by default, /root, /etc, /bin, and /usr are vnddirs)
Some future version of the script will allow you to define subdirectories of a vnddir as a separate paritition in vnddirs (such as /usr/src mounted on top of /usr.) For today, /usr can only be a single partition.
flashrd is designed to accommodate NSH without much hassle. NSH is intended to control configuration of most major daemons and functions for networking (routers, firewalls, VPN appliances). This is a major configuration shift and must be handled with care.The suggested installation procedure:
Unpack and place nsh in /flash
Place nsh in /flash to avoid overwriting during upgrade flashrd system upgrade procedurecp nsh-XXXXXXXX.tar.gz /flash
cd /flash
tar xzf nsh-XXXXXXXX.tar.gz
cd nsh
make LDADD="-ledit -ltermcap -static" CFLAGS="-O -DDHCPLEASES=\"/flash/dhcpd.leases\""
This will force NSH to be built as a static binary, so that you may upgrade the flashrd system without affecting nsh. Also it forces nsh to store dhcpd leases in /flash/dhcpd.leases (instead of /var/db, to preserve lease persistency across flashrd reboots). It is suggested you re-build nsh from a matching system (matching version and architecture) during upgrades. If you can't do this, building NSH as a static binary helps, as there are typically not changes made between OpenBSD versions that will break NSH to the point where you can't remotely login!
Enter read/write mode
rwActivate save.sh to save NSH configurations to /flash
NSH calls /usr/local/bin/save.sh to permanently store configurations. It does this because you may want to run other commands at the same time that you store the config (such as check the config into a revision control system, or to read/write, read/only state on a partition other than /flash, or whatever else you can imagine.)sed -e 's/etc/flash/' < save-rw.sh > /usr/local/bin/save.sh
chmod a+x /usr/local/bin/save.sh
Run nsh on startup
echo "/flash/nsh/nsh -i /flash/nshrc" >> /etc/rc.local
echo "sysctl ddb.panic=0" >> /etc/rc.local
Remove manual network configurations
rm /etc/hostname.vr0
rm /etc/mygate
Also be sure to stop any other daemons (that you intend to control via NSH) from starting up before NSH
echo sshd_flags=NO >>/etc/rc.conf.local
echo pf=NO >>/etc/rc.conf.local
echo inetd=NO >>/etc/rc.conf.local
Link nsh to /bin
ln -s /flash/nsh/nsh /bin/nshCreate your initial NSH configuration
Upon 'write', nsh will automatically grab interface IPs, aliases, and routes from the kernel. NSH looks in:
/var/run/pf.conf
/var/run/ospfd.conf
/var/run/bgpd.conf
/var/run/ripd.conf
/var/run/ipsec.conf
/var/run/dvmrpd.conf
/var/run/relayd.conf
/var/run/sasyncd.conf
/var/run/dhcpd.conf
/var/run/snmpd.conf
/var/run/ntpd.conf
/var/run/resolv.conf
/var/run/inetd.conf
/var/run/sshd.conf
(when you 'write' a new config to disk) to store daemon configurations persistently inside the nshrc.With this knowledge, if you wanted to control PF, DHCP and the local resolver through NSH, here's what it would look like:
cp /etc/pf.conf /var/run/pf.conf
cp /etc/dhcpd.conf /var/run/dhcpd.conf
mv /etc/resolv.conf /var/run/resolv.conf
ln -s /var/run/resolv.conf.symlink /etc/resolv.conf
nsh
nsh/enable
nsh/pf enable
nsh/dhcp enable
nsh/dns local-control
nsh/write
nsh/quit
Resume read-only mode
roNow you must review /flash/nshrc to see if NSH has everything you need. Look for interface IPs being properly configured, routes being in place, BGP, OSPF, DHCP, etc... configuration being in place (and the daemons enabled), the goal here is to boot up with nsh and expect that you can manage all daemons through it!
Now it's time to reboot your flashrd system and watch nsh take over boot-up configurations, to ensure proper installation. If you haven't used nsh before, it is strongly suggested that you are on-site (or have remote serial console access) should nsh not do what you expect it to do!
You can obtain new image files by building or downloading a new flashrd image, mounting the vnd file, and copying them out of the mounted partition.gzip -d flashimg.i386-20120909.gz
vnconfig svnd0 flashimg.i386-20120909
mount /dev/svnd0a /mnt
cd /mnt
cp bsd openbsd.vnd var.tar /tmp
cd /tmp
umount /mnt
vnconfig -u svnd0
You could also try running mkkern and mkdist directly to obtain the raw FLASHRD bsd kernel, openbsd.vnd, and var.tar files. mkkern leaves output in a directory named with the format: /tmp/mkkern.XXXXXXXX while mkdist creates openbsd.vnd and var.tar in your current working directory under a new subdirectory vnd-YYYYMMDD.
On the flash boot image (which is mounted as /flash when booted under flashrd) you have three image files that define the whole system (well, whole system except the boot blocks and bootstrap script):bsd
openbsd.vnd
var.tar
This is the layout accoring to the default configuration that defines 'var' as handled under 'tardirs'. If you moved var to 'vnddirs' then you would only have a flashrd ramdisk kernel and openbsd.vnd file.
To upgrade, first you need to run 'cfgflashrd' against the flash boot image that you are upgrading to. This is important as cfgflashrd will setup both the bootstrap script (part of the ramdisk kernel) to the proper disk type (if you specify sd0 instead of wd0 for USB keys, perhaps), and will setup the /etc/boot.conf and /etc/ttys files for serial console use (if you activate com0).
If the wd0 image default disk type is fine for your system, you can skip running cfgflashrd against the new flash image and instead simply copy /etc/ttys and /etc/boot.conf from the source /etc to the new /etc directory (like you see below for hostname.vr0 and mygate.)
Once you have a properly configured image that reflects the disk type and console type of your destination system, you need to extract the above mentioned image files and upload the new image files to /flash/new, modify the new /etc directory to reflect your needs, move the old files into /flash/old, move the /flash/new files to /flash, and then reboot. This might look something like:
Modify new /etc to include necessary configuration (non-NSH case)
Here we assume /etc is partition e, as is the default case with flashrd. Also we assume vr0 is your primary network card. Naturally you will want to copy configuration for the appropriate network adapter, and include any other configuration files that you want to carry over to the new system.vnconfig svnd1 /flash/new/openbsd.vnd
mount /dev/svnd1e /mnt
cp /etc/hostname.vr0 /mnt/
cp /etc/mygate /mnt/
cp /etc/ssh/ssh_host_* /mnt/ssh/
umount /mnt
vnconfig -u svnd1
If you forget to copy a config over, you can later retrieve it from the /etc partition of the vnd image stored in /flash/old/openbsd.vnd
Modify new /etc to include necessary configuration (NSH case)
Here we assume that you start nsh from /etc/rc.local and that you initially installed NSH according to the directions above. We also assume that /etc is partition e, as is the default case with flashrd.vnconfig svnd1 /flash/new/openbsd.vnd
mount /dev/svnd1e /mnt
cp /etc/rc.local /mnt/
cp /etc/rc.conf.local /mnt/
cp /etc/ssh/ssh_host_* /mnt/ssh/
umount /mnt
vnconfig -u svnd1
Move new FLASHRD kernel/system image in place
cd /flash
mv bsd openbsd.vnd var.tar /flash/old
cd /flash/new
mv bsd openbsd.vnd var.tar /flash
Boot new image
reboot
If fsck -y or mount fails to mount any partition within /flash/openbsd.vnd (your default image), the bootstrap script will try to run with the /flash/old/openbsd.vnd image instead. Read /stand/rc to see exactly what happens. (It is fairly simplistic today, and I'm sure it could be improved upon.)
Stop using bash. Executing flashrd from bash seems to be problematic. OpenBSD's ksh has the features that most people use bash for anyways, such as command line history recall / editing and ksh's programmability (which gets attributed to bash by new comers.) Try it out, you'll find that installing bash is superfluous (and, at least in this case, harmful.)
I've had several compact flash ATA disks (mounted in read-write mode) get scrambled for no apparent reason (under multiple operating systems!) The issues that did not appear to be wear related (no bad sectors on read), and these systems did not often write to flash. Read-only mounts avoid some corruption that could otherwise happen. (It's also a basic preventitive measure to avoid dirty filesystems that fsck -p can't fix) The objective of flashrd is to create a system that runs reliably for many years, with a minimal chance of failure.
Copy your preferred time description file to /etc/localtime. Either look in /usr/share/zoneinfo for the right file or just copy /etc/localtime over from a box that already has the right timezone.
Pkg tools work as expected under flashrd just like they do on a default OpenBSD install. The process is identical to a normal system, except you have to go to rw mode before installing, and we suggest going back to ro mode after. (Actually run the commands 'rw' and 'ro' to do this under flashrd!)