Rebuild the options files of FreeBSD ports

FreeBSD,ports,options,pkg,rebuild

I noticed that the /var/db/ports/ directory on my FreeBSD machine, which I’ve been running for about 15 years, contains a mix of old and new ports option information. Recently (that is, since June 2013), the directory names here have followed the format "CategoryName_PortsName" (e.g., x11_xorg-apps). It seems the directory names didn’t include the category name in the past, and those old entries have remained there ever since.

I thought I’d used a script to completely delete the old-style directories, but somehow, due to a mistake, /var/db/ports itself ended up getting wiped out. Oh no!

Since I didn’t have any backups, I decided to rebuild /var/db/ports based on the information in the pkg database of currently installed packages. In theory, this should be possible, so I asked ChatGPT to explain the following specifications and help me out.

• Any entries in the `pkg info` output that include `built_by` are excluded from restoration because they originate from binary packages.
• Among the candidates for restoration, only those for which `origin` exists in the current ports tree and that actually have `OPTIONS` are processed.
• The restoration results are created in a temporary directory and can be verified using `make PORT_DBDIR=/tmp/zzz showconfig`.

I designed the script to perform the above processes in two stages.

• Step 1: Create a list of items to be restored.（step1-create-recover-list.sh）
• Step 2: Read the list of items to be restored and restore the contents of /var/db/ports/ (options files) to /tmp/recovered-ports/ (step2-recover-options.sh）

Step 1

When you run the script in Step 1, the following files will be generated.

• recover.list：A file listing the origins present in the current ports tree that are candidates for restoration
• recover-packages.tsv：Details of the packages selected as candidates for restoration
• binary-packages.tsv：Packages excluded because they have a "built_by" entry
• missing-origins.tsv：Candidates for restoration for which no origin exists in the current ports tree
• no-origin-packages.tsv：Packages with an empty or invalid origin

This script marks all packages that lack a `built_by` field as candidates for restoration. Therefore, the `recover.list` also includes the following:

• ports that were explicitly installed from ports tree
• ports that were automatically built from ports tree as dependencies
• Items for which the `built_by` entry is not recorded for some reason, such as older binary packages

On the other hand, since users may have modified the OPTIONS file for dependent ports as well, Step 1 does not exclude those with `automatic=1`. In the `recover-packages.tsv` file, a value of 1 in the last column indicates a package that was automatically registered as a dependency, while a value of 0 indicates a package that was explicitly installed.

Step 2

In Step 2, the following processing is performed for each origin. The input is the `recover.list` file generated in Step 1.

1. Retrieve OPTIONS_NAME, COMPLETE_OPTIONS_LIST, and the default PORT_OPTIONS from the current port
2. Retrieve build-time OPTIONS from the %Ok (OPTIONS name) and %Ov (OPTIONS value) entries for installed packages
3. For OPTIONS that still exist, prioritize the values from the package records
4. For OPTIONS added after the package was built, use the current default values
5. Do not restore old OPTIONS that have been deleted or renamed; log them instead
6. Generate in /tmp/recovered-ports instead of /var/db/ports

When you run the script in Step 2, the following log files are generated.

• recovered.tsv
• no-options.tsv
• no-package-options.tsv
• obsolete-options.tsv
• new-options.tsv
• conflicts.tsv
• failed.tsv

Confirmation

Perform a batch verification to ensure that the ports framework can successfully read the options files for the generated origin.

If the output of this script shows "Validation failures: 0," you can conclude that the syntax and consistency with the current ports tree have been verified.

When actually verifying the settings for `editors/emacs`, for example, do not assume the value of `OPTIONS_NAME` directly; instead, check it as follows.

cd /usr/ports/editors/emacs

make PORT_DBDIR=/tmp/recovered-ports -V OPTIONS_NAME
make PORT_DBDIR=/tmp/recovered-ports showconfig

Installation

Finally, install it into an empty `/var/db/ports` directory.

install -d -m 755 /var/db/ports
cp -Rp /tmp/recovered-ports/. /var/db/ports/

Conclusion

Since this is just optional information for ports, it’s fine if you don’t have it, but I figured I’d go ahead and restore it from pkg anyway. Please use this only if you’re having trouble because you accidentally deleted /var/db/ports/ in the same way. Please check everything thoroughly before actually using these scripts. I take absolutely no responsibility for any problems that may arise in your environment.

Atsushi NUNOME @ Computer System Laboratory at Kyoto Institute of Technology (nunomekit.ac.jp)