Install FreeBSD.
Create a user to own the portbuild
repository, such as portbuild
. It should have the
'*'
password.
Similarly, create a user to own the administration functions
and manage the svn
repositories, such as srcbuild
. It should have the
'*'
password.
Add the following to /boot/loader.conf
:
console="vidconsole,comconsole"
You should run the cluster on UTC. If you have not set the clock to UTC:
#
cp -p /usr/share/zoneinfo/Etc/UTC /etc/localtime
Create the appropriate
/etc/rc.conf
.
Required entries:
hostname="${hostname}
"
sshd_enable="YES"
zfs_enable="YES"
Recommended entries:
background_fsck="NO" clear_tmp_enable="YES" dumpdev="AUTO" fsck_y_enable="YES" apache22_enable="YES" apache_flags="" apache_pidfile="/var/run/httpd.pid" inetd_enable="YES" inetd_flags="-l -w" mountd_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 12" nfs_remote_port_only="YES" ntpd_enable="YES" rpcbind_enable="YES" rpc_lockd_enable="NO" rpc_statd_enable="YES" sendmail_enable="NONE" smartd_enable="YES"
If you are using ganglia, add:
gmetad_enable="YES" gmond_enable="YES"
If you will be using a squid cache on the server, rather than the clients:
squid_enable="YES"
Create /etc/resolv.conf
, if
necessary.
Create the appropriate files in
/etc/ssh/
.
Add the following to /etc/sysctl.conf
:
kern.maxfiles=40000 kern.maxfilesperproc=38000 sysctl vfs.usermount=1 sysctl vfs.zfs.super_owner=1
Make sure the following change is made to
/etc/ttys
:
ttyu0 "/usr/libexec/getty std.9600" vt100 on secure
You should be able to install from the most recent release using only the default kernel configuration.
The following ports (or their latest successors) are required:
databases/py-sqlite3 databases/py-sqlalchemy (only SQLITE is needed) devel/git (WITH_SVN) devel/py-configobj devel/py-setuptools devel/subversion net/nc net/rsync www/apache22 (with EXT_FILTER)
Expect those to bring in, among others:
databases/sqlite3 lang/perl-5.14 (or successor) lang/python27 (or sucessor)
If you are using ganglia, add:
sysutils/ganglia-monitor-core (with GMETAD off) sysutils/ganglia-webfrontend (compile with -DWITHOUT_X11)
If you will be using a squid cache on the server, rather than the clients:
www/squid (with SQUID_AUFS on)
The following ports (or their latest successors) are strongly suggested:
devel/ccache mail/postfix net/isc-dhcp41-server ports-mgmt/pkg ports-mgmt/portaudit ports-mgmt/portmaster shells/bash shells/zsh sysutils/screen
The use of sudo on the master, which was formerly required, is no longer recommended.
The following ports (or their latest successors) are handy:
benchmarks/bonnie++ ports-mgmt/pkg_tree sysutils/dmidecode sysutils/smartmontools sysutils/zfs-stats
The following steps need to be done as euid root.
Here is a quick example:
portbuild/tools/example_install
#!/bin/sh # # example script to drive the "mkportbuild" kickstart file # export PORTBUILD_USER=portbuild export SRCBUILD_USER=srcbuild export ZFS_VOLUME=a export ZFS_MOUNTPOINT=/a export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org # # create the zpool. the examples here are just suggestions and need to be # customized for your site. # # simple examples: # zpool create ${ZFS_VOLUME} da1 # zpool create ${ZFS_VOLUME} gprootfs # more complex example: # zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8 # # check out the kickstart file and run it # mkdir -p tmp svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/admin/tools tmp sh -x ./tmp/mkportbuild
Here is a detailed explanation of the example:
Export the value of PORTBUILD_USER
:
#
export PORTBUILD_USER=portbuild
Export the value of SRCBUILD_USER
:
#
export SRCBUILD_USER=srcbuild
Pick a zfs volume name and export
it. We have used a
so far to date.
#
export ZFS_VOLUME=a
Pick a mountpoint and export it. We have used
/
so far to date.a
#
export ZFS_MOUNTPOINT=/a
Create the zfs volume and mount it.
#
zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
The kickstart script defines zfs
permission sets
, so that the
srcbuild
user and
portbuild
user may administer
subdirectories of this
volume without having to have root privileges.
Select an svn repository and export it. See the FreeBSD Handbook for the currently supported list.
#
export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org
Obtain a copy of the kickstart script into a temporary directory. (You will not need to keep this directory later.)
#
mkdir -p /home/portbuild
/tmp
#
svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/admin/tools /home/portbuild
/tmp
Run the kickstart script:
#
sh /home/portbuild
/tmp
/mkportbuild
This will accomplish all the following steps:
Create the portbuild
directory
Create and mount a new zfs filesystem on it
Set up the directory
Set up the initial repository:
Set up the zfs
permission sets
.
Split ownerships of subdirectories such that
PORTBUILD_USER
owns, and
only owns, files that are used to manage builds and
interact with slaves. The more trustable user
SRCBUILD_USER
now owns
everything else.
Configure the server by making the following changes to
/
:a
/portbuild/admin/conf/admin.conf
Set SUPPORTED_ARCHS
to the
list of architectures you wish to build packages for.
For each source branch you will be building for, set
SRC_BRANCHES
and
SRC_BRANCH_
as detailed in Section 14.1, “Steps necessary before qmanager is started”.
You should not need to change
branch
_SUBDIRSRC_BRANCHES_PATTERN
.
Set ZFS_VOLUME
and
ZFS_MOUNTPOINT
to whatever you
chose above.
Set VCS_REPOSITORY
to whatever
you chose above.
Set MASTER_URL
to the http
URL of your server. This will be stamped into the
package build logs and the indices thereof.
Most of the other default values should be fine.
Configure how build slaves will talk to your server
by making the following changes to
/
:a
/portbuild/conf/client.conf
Set CLIENT_NFS_MASTER
to wherever
your build slaves will PXE boot from. (Possibly, the
hostname of your server.)
Set CLIENT_BACKUP_FTP_SITE
to a backup site for FTP fetches; again, possibly
the hostname of your server.
Set CLIENT_UPLOAD_HOST
to
where completed packages will be uploaded.
Most of the other default values should be fine.
Most of the default values in
/
should be fine. This file holds definitions used by
both the server and all its clients.a
/portbuild/conf/common.conf
Configure the server by making the following changes to
/
:a
/portbuild/conf/server.conf
Set UPLOAD_DIRECTORY
,
UPLOAD_TARGET
, and
UPLOAD_USER
as appropriate
for your site.
Most of the other default values should be fine.
For each architecture, follow the steps in Section 17.1, “Steps necessary before qmanager is started”.
As root
, copy the following files from
/a/portbuild/admin/etc/rc.d/
to
/usr/local/etc/rc.d/
:
pollmachine qmanager
As root, start each one of them. You may find it handy to start each under screen for debugging purposes.
Initialize the qmanager database's acl list:
This should now be automatically done for you by
the first build
command.
#
python /a
/portbuild/qmanager/qclient add_acl name=deny_all uidlist= gidlist= sense=0
As the srcbuild
user,
run the following commands manually to create the
src
and ports
repositories, respectively:
%
/a
/portbuild/admin/scripts/updatesnap.ports%
/a
/portbuild/admin/scripts/updatesnap
These will be periodically run from the
srcbuild
crontab
, which you will
install below.
Configure
/usr/local/etc/apache22/httpd.conf
as appropriate for your site.
Copy /a/portbuild/admin/conf/apache.conf
to the appropriate Includes/
subdirectory, e.g.,
/usr/local/etc/apache22/Includes/portbuild.conf
.
Configure it as appropriate for your site.
Install /a/portbuild/admin/crontabs/portbuild
as
the portbuild
crontab via
crontab -u portbuild -e
. If you do
not support all the archs listed there, make sure to comment out
the appropriate dologs entries.
Install /a/portbuild/admin/crontabs/srcbuild
as
the srcbuild
crontab via
crontab -u srcbuild -e
.
If your build slaves will be pxebooted, make sure to
enable the tftp entries in
/etc/inetd.conf
.
Configure mail by doing the following:
newaliases
.
For each architecture, follow the steps in Section 17.2, “Steps necessary after qmanager is started”.
You will probably find it handy to append
the following to the PATH
definition for
the portbuild
user:
/a
/portbuild/scripts:/a
/portbuild/tools
You will also probably find it handy to append
the following to the PATH
definition for
the srcbuild
user:
/a
/portbuild/admin/scripts:/a
/portbuild/admin/tools
You should now be ready to build packages.
All FreeBSD documents are available for download at http://ftp.FreeBSD.org/pub/FreeBSD/doc/
Questions that are not answered by the
documentation may be
sent to <freebsd-questions@FreeBSD.org>.
Send questions about this document to <freebsd-doc@FreeBSD.org>.