                        FreeBSD and Solid State Devices

  John Kozubik

   <john@kozubik.com>

   Revision: 43126

   Copyright (c) 2001, 2009 The FreeBSD Documentation Project

   FreeBSD is a registered trademark of the FreeBSD Foundation.

   Many of the designations used by manufacturers and sellers to distinguish
   their products are claimed as trademarks. Where those designations appear
   in this document, and the FreeBSD Project was aware of the trademark
   claim, the designations have been followed by the "(TM)" or the "(R)"
   symbol.

   Copyright

   Redistribution and use in source (XML DocBook) and 'compiled' forms (XML,
   HTML, PDF, PostScript, RTF and so forth) with or without modification, are
   permitted provided that the following conditions are met:

    1. Redistributions of source code (XML DocBook) must retain the above
       copyright notice, this list of conditions and the following disclaimer
       as the first lines of this file unmodified.

    2. Redistributions in compiled form (transformed to other DTDs, converted
       to PDF, PostScript, RTF and other formats) must reproduce the above
       copyright notice, this list of conditions and the following disclaimer
       in the documentation and/or other materials provided with the
       distribution.

  Important:

   THIS DOCUMENTATION IS PROVIDED BY THE FREEBSD DOCUMENTATION PROJECT "AS
   IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
   THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
   PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FREEBSD DOCUMENTATION
   PROJECT BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
   EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
   PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
   PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
   LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
   NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
   DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

   Last modified on 2013-11-07 by gabor.
   Abstract

   This article covers the use of solid state disk devices in FreeBSD to
   create embedded systems.

   Embedded systems have the advantage of increased stability due to the lack
   of integral moving parts (hard drives). Account must be taken, however,
   for the generally low disk space available in the system and the
   durability of the storage medium.

   Specific topics to be covered include the types and attributes of solid
   state media suitable for disk use in FreeBSD, kernel options that are of
   interest in such an environment, the rc.initdiskless mechanisms that
   automate the initialization of such systems and the need for read-only
   filesystems, and building filesystems from scratch. The article will
   conclude with some general strategies for small and read-only FreeBSD
   environments.

     ----------------------------------------------------------------------

   Table of Contents

   1. Solid State Disk Devices

   2. Kernel Options

   3. The rc Subsystem and Read-Only Filesystems

   4. Building a File System From Scratch

   5. System Strategies for Small and Read Only Environments

1. Solid State Disk Devices

   The scope of this article will be limited to solid state disk devices made
   from flash memory. Flash memory is a solid state memory (no moving parts)
   that is non-volatile (the memory maintains data even after all power
   sources have been disconnected). Flash memory can withstand tremendous
   physical shock and is reasonably fast (the flash memory solutions covered
   in this article are slightly slower than a EIDE hard disk for write
   operations, and much faster for read operations). One very important
   aspect of flash memory, the ramifications of which will be discussed later
   in this article, is that each sector has a limited rewrite capacity. You
   can only write, erase, and write again to a sector of flash memory a
   certain number of times before the sector becomes permanently unusable.
   Although many flash memory products automatically map bad blocks, and
   although some even distribute write operations evenly throughout the unit,
   the fact remains that there exists a limit to the amount of writing that
   can be done to the device. Competitive units have between 1,000,000 and
   10,000,000 writes per sector in their specification. This figure varies
   due to the temperature of the environment.

   Specifically, we will be discussing ATA compatible compact-flash units,
   which are quite popular as storage media for digital cameras. Of
   particular interest is the fact that they pin out directly to the IDE bus
   and are compatible with the ATA command set. Therefore, with a very simple
   and low-cost adaptor, these devices can be attached directly to an IDE bus
   in a computer. Once implemented in this manner, operating systems such as
   FreeBSD see the device as a normal hard disk (albeit small).

   Other solid state disk solutions do exist, but their expense, obscurity,
   and relative unease of use places them beyond the scope of this article.

2. Kernel Options

   A few kernel options are of specific interest to those creating an
   embedded FreeBSD system.

   All embedded FreeBSD systems that use flash memory as system disk will be
   interested in memory disks and memory filesystems. Because of the limited
   number of writes that can be done to flash memory, the disk and the
   filesystems on the disk will most likely be mounted read-only. In this
   environment, filesystems such as /tmp and /var are mounted as memory
   filesystems to allow the system to create logs and update counters and
   temporary files. Memory filesystems are a critical component to a
   successful solid state FreeBSD implementation.

   You should make sure the following lines exist in your kernel
   configuration file:

 options         MFS             # Memory Filesystem
 options         MD_ROOT         # md device usable as a potential root device
 pseudo-device   md              # memory disk

3. The rc Subsystem and Read-Only Filesystems

   The post-boot initialization of an embedded FreeBSD system is controlled
   by /etc/rc.initdiskless.

   /etc/rc.d/var mounts /var as a memory filesystem, makes a configurable
   list of directories in /var with the mkdir(1) command, and changes modes
   on some of those directories. In the execution of /etc/rc.d/var, one other
   rc.conf variable comes into play - varsize. The /etc/rc.d/var file creates
   a /var partition based on the value of this variable in rc.conf:

 varsize=8192

   Remember that this value is in sectors by default.

   The fact that /var is a read-write filesystem is an important distinction,
   as the / partition (and any other partitions you may have on your flash
   media) should be mounted read-only. Remember that in Section 1, "Solid
   State Disk Devices" we detailed the limitations of flash memory -
   specifically the limited write capability. The importance of not mounting
   filesystems on flash media read-write, and the importance of not using a
   swap file, cannot be overstated. A swap file on a busy system can burn
   through a piece of flash media in less than one year. Heavy logging or
   temporary file creation and destruction can do the same. Therefore, in
   addition to removing the swap entry from your /etc/fstab file, you should
   also change the Options field for each filesystem to ro as follows:

 # Device                Mountpoint      FStype  Options         Dump    Pass#
 /dev/ad0s1a             /               ufs     ro              1       1

   A few applications in the average system will immediately begin to fail as
   a result of this change. For instance, cron will not run properly as a
   result of missing cron tabs in the /var created by /etc/rc.d/var, and
   syslog and dhcp will encounter problems as well as a result of the
   read-only filesystem and missing items in the /var that /etc/rc.d/var has
   created. These are only temporary problems though, and are addressed,
   along with solutions to the execution of other common software packages in
   Section 5, "System Strategies for Small and Read Only Environments".

   An important thing to remember is that a filesystem that was mounted
   read-only with /etc/fstab can be made read-write at any time by issuing
   the command:

 # /sbin/mount -uw partition

   and can be toggled back to read-only with the command:

 # /sbin/mount -ur partition

4. Building a File System From Scratch

   Because ATA compatible compact-flash cards are seen by FreeBSD as normal
   IDE hard drives, you could theoretically install FreeBSD from the network
   using the kern and mfsroot floppies or from a CD.

   However, even a small installation of FreeBSD using normal installation
   procedures can produce a system in size of greater than 200 megabytes.
   Because most people will be using smaller flash memory devices (128
   megabytes is considered fairly large - 32 or even 16 megabytes is common)
   an installation using normal mechanisms is not possible-there is simply
   not enough disk space for even the smallest of conventional installations.

   The easiest way to overcome this space limitation is to install FreeBSD
   using conventional means to a normal hard disk. After the installation is
   complete, pare down the operating system to a size that will fit onto your
   flash media, then tar the entire filesystem. The following steps will
   guide you through the process of preparing a piece of flash memory for
   your tarred filesystem. Remember, because a normal installation is not
   being performed, operations such as partitioning, labeling, file-system
   creation, etc. need to be performed by hand. In addition to the kern and
   mfsroot floppy disks, you will also need to use the fixit floppy.

    1. Partitioning your flash media device

       After booting with the kern and mfsroot floppies, choose custom from
       the installation menu. In the custom installation menu, choose
       partition. In the partition menu, you should delete all existing
       partitions using the d key. After deleting all existing partitions,
       create a partition using the c key and accept the default value for
       the size of the partition. When asked for the type of the partition,
       make sure the value is set to 165. Now write this partition table to
       the disk by pressing the w key (this is a hidden option on this
       screen). If you are using an ATA compatible compact flash card, you
       should choose the FreeBSD Boot Manager. Now press the q key to quit
       the partition menu. You will be shown the boot manager menu once more
       - repeat the choice you made earlier.

    2. Creating filesystems on your flash memory device

       Exit the custom installation menu, and from the main installation menu
       choose the fixit option. After entering the fixit environment, enter
       the following command:

 # disklabel -e /dev/ad0c

       At this point you will have entered the vi editor under the auspices
       of the disklabel command. Next, you need to add an a: line at the end
       of the file. This a: line should look like:

 a:      123456  0       4.2BSD  0       0

       Where 123456 is a number that is exactly the same as the number in the
       existing c: entry for size. Basically you are duplicating the existing
       c: line as an a: line, making sure that fstype is 4.2BSD. Save the
       file and exit.

 # disklabel -B -r /dev/ad0c
 # newfs /dev/ad0a

    3. Placing your filesystem on the flash media

       Mount the newly prepared flash media:

 # mount /dev/ad0a /flash

       Bring this machine up on the network so we may transfer our tar file
       and explode it onto our flash media filesystem. One example of how to
       do this is:

 # ifconfig xl0 192.168.0.10 netmask 255.255.255.0
 # route add default 192.168.0.1

       Now that the machine is on the network, transfer your tar file. You
       may be faced with a bit of a dilemma at this point - if your flash
       memory part is 128 megabytes, for instance, and your tar file is
       larger than 64 megabytes, you cannot have your tar file on the flash
       media at the same time as you explode it - you will run out of space.
       One solution to this problem, if you are using FTP, is to untar the
       file while it is transferred over FTP. If you perform your transfer in
       this manner, you will never have the tar file and the tar contents on
       your disk at the same time:

 ftp> get tarfile.tar "| tar xvf -"

       If your tarfile is gzipped, you can accomplish this as well:

 ftp> get tarfile.tar "| zcat | tar xvf -"

       After the contents of your tarred filesystem are on your flash memory
       filesystem, you can unmount the flash memory and reboot:

 # cd /
 # umount /flash
 # exit

       Assuming that you configured your filesystem correctly when it was
       built on the normal hard disk (with your filesystems mounted
       read-only, and with the necessary options compiled into the kernel)
       you should now be successfully booting your FreeBSD embedded system.

5. System Strategies for Small and Read Only Environments

   In Section 3, "The rc Subsystem and Read-Only Filesystems", it was pointed
   out that the /var filesystem constructed by /etc/rc.d/var and the presence
   of a read-only root filesystem causes problems with many common software
   packages used with FreeBSD. In this article, suggestions for successfully
   running cron, syslog, ports installations, and the Apache web server will
   be provided.

  5.1. cron

   Upon boot, /var gets populated by /etc/rc.d/var using the list from
   /etc/mtree/BSD.var.dist, so the cron, cron/tabs, at, and a few other
   standard directories get created.

   However, this does not solve the problem of maintaining cron tabs across
   reboots. When the system reboots, the /var filesystem that is in memory
   will disappear and any cron tabs you may have had in it will also
   disappear. Therefore, one solution would be to create cron tabs for the
   users that need them, mount your / filesystem as read-write and copy those
   cron tabs to somewhere safe, like /etc/tabs, then add a line to the end of
   /etc/rc.initdiskless that copies those crontabs into /var/cron/tabs after
   that directory has been created during system initialization. You may also
   need to add a line that changes modes and permissions on the directories
   you create and the files you copy with /etc/rc.initdiskless.

  5.2. syslog

   syslog.conf specifies the locations of certain log files that exist in
   /var/log. These files are not created by /etc/rc.d/var upon system
   initialization. Therefore, somewhere in /etc/rc.d/var, after the section
   that creates the directories in /var, you will need to add something like
   this:

 # touch /var/log/security /var/log/maillog /var/log/cron /var/log/messages
 # chmod 0644 /var/log/*

  5.3. Ports Installation

   Before discussing the changes necessary to successfully use the ports
   tree, a reminder is necessary regarding the read-only nature of your
   filesystems on the flash media. Since they are read-only, you will need to
   temporarily mount them read-write using the mount syntax shown in
   Section 3, "The rc Subsystem and Read-Only Filesystems". You should always
   remount those filesystems read-only when you are done with any maintenance
   - unnecessary writes to the flash media could considerably shorten its
   lifespan.

   To make it possible to enter a ports directory and successfully run make
   install, we must create a packages directory on a non-memory filesystem
   that will keep track of our packages across reboots. Because it is
   necessary to mount your filesystems as read-write for the installation of
   a package anyway, it is sensible to assume that an area on the flash media
   can also be used for package information to be written to.

   First, create a package database directory. This is normally in
   /var/db/pkg, but we cannot place it there as it will disappear every time
   the system is booted.

 # mkdir /etc/pkg

   Now, add a line to /etc/rc.d/var that links the /etc/pkg directory to
   /var/db/pkg. An example:

 # ln -s /etc/pkg /var/db/pkg

   Now, any time that you mount your filesystems as read-write and install a
   package, the make install will work, and package information will be
   written successfully to /etc/pkg (because the filesystem will, at that
   time, be mounted read-write) which will always be available to the
   operating system as /var/db/pkg.

  5.4. Apache Web Server

  Note:

   The steps in this section are only necessary if Apache is set up to write
   its pid or log information outside of /var. By default, Apache keeps its
   pid file in /var/run/httpd.pid and its log files in /var/log.

   It is now assumed that Apache keeps its log files in a directory
   apache_log_dir outside of /var. When this directory lives on a read-only
   filesystem, Apache will not be able to save any log files, and may have
   problems working. If so, it is necessary to add a new directory to the
   list of directories in /etc/rc.d/var to create in /var, and to link
   apache_log_dir to /var/log/apache. It is also necessary to set permissions
   and ownership on this new directory.

   First, add the directory log/apache to the list of directories to be
   created in /etc/rc.d/var.

   Second, add these commands to /etc/rc.d/var after the directory creation
   section:

 # chmod 0774 /var/log/apache
 # chown nobody:nobody /var/log/apache

   Finally, remove the existing apache_log_dir directory, and replace it with
   a link:

 # rm -rf apache_log_dir
 # ln -s /var/log/apache apache_log_dir
