Marius van Witzenburg » FreeBSD

How to easy CVS update your SRC and Ports tree with a simple Bash script (v2)

mariusvw — Mon, 26 Sep 2011 20:29:22 +0000

This is an update to this post: How to easy CVS update your SRC and Ports tree with a simple Bash script

I adjusted the script to use the local csup version of the FreeBSD World system and moved the cvsup files to /etc.

First install a simple tool to get the fastest cvs server

pkg_add -r fastest_cvsup

Or from Ports (if you have the collection already):

cd /usr/ports/sysutils/fastest_cvsup/
make install clean

Setup config and binary to easy update the repository for SRC and PORTS.

mkdir /etc/cvsup
cd /usr/share/examples/cvsup
cp stable-supfile /etc/cvsup
cp ports-supfile /etc/cvsup
cp doc-supfile /etc/cvsup
cd /etc/cvsup

Edit stable-supfile and change:

*default release=cvs tag=RELENG_8_2

Set this to your release version.

Edit /usr/local/sbin/runcvsup and add:

#!/bin/sh
arg=$@
sType="tld,nl" # Can be: all / tld / tld,nl,de,fr,us
if SERVER=`/usr/local/bin/fastest_cvsup -q -c $sType`; then
    echo "Using server: ${SERVER}"
    case "$arg" in
    "stable")
        echo "Updating: stable"
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/stable-supfile
    ;;
    "ports")
        echo "Updating: ports"
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/ports-supfile
    ;;
    "doc")  
        echo "Updating: doc"
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/doc-supfile
    ;;
    *)
        echo "Updating: all"
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/stable-supfile
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/ports-supfile
        /usr/bin/csup -L 2 -g -z -h ${SERVER} /etc/cvsup/doc-supfile
    ;;
    esac    
fi

Set the right permissions:

chmod 700 /usr/local/sbin/runcvsup

Update everything:

runcvsup

How to update your FreeBSD kernel and World system

mariusvw — Sun, 10 Jul 2011 15:34:27 +0000

Auto install using freebsd-update

See: How to use freebsd-update to update your FreeBSD kernel and world system

Manual Update World / Kernel

Its advised to run this in Screen, you can install it from /usr/ports/sysutils/screen.

First you have to update the SRC and Ports tree, follow the manual on this page to complete this step.

Next is to configure the make.conf so it won't install things we don't want.
Be sure you check out man make.conf for some make optimalisation settings.

Here is an example that I use:

WITHOUT_X11=yes
NO_GAMES=yes
NO_X=yes
 
# make optimalization (-j4, 4 = optimal for 1 cpu)
MAKEOPTS="-j4 -B"

In case you have edited (which is a 100% chance you did) files of the world installation you might need to edit the mergemaster.rc file and add some settings before running mergemaster.

Check this page to view an example: How to configure mergemaster to merge configurations on FreeBSD.

These are the steps to install a fresh kernel and world system. The TMPDIR that I use is because I disabled the ability to execute files on the standard /tmp mount.

mkdir /tmp_world
setenv TMPDIR /tmp_world
cd /usr/src
make clean
make buildworld
make buildkernel
make installkernel
make installworld

Merge the world system:

mergemaster

Be sure you check out man mergemaster. For example people use the -p argument. This can be handy in some situations.

Remove temp dir:

unsetenv TMPDIR
rm -r /tmp_world

Short versions (might be unsafe):

# Kernel & World
mkdir /tmp_world ; setenv TMPDIR /tmp_world; cd /usr/src; make clean
make buildworld && make buildkernel && make installkernel && make installworld
mergemaster
unsetenv TMPDIR; rm -r /tmp_world
 
# Kernel-only
mkdir /tmp_world ; setenv TMPDIR /tmp_world; cd /usr/src; make clean
make buildkernel; make installkernel
unsetenv TMPDIR; rm -r /tmp_world

Remove builded objects, only after a release upgrade:

cd /usr/obj
chflags -R noschg *
rm -rf *

Custom KERNEL

cd /usr/src/sys/i386/conf/
cp GENERIC YOURKERNEL
echo "# Kernel configuration" >> /etc/make.conf
echo "KERNCONF=YOURKERNEL" >> /etc/make.conf

Compile the same as above, only leave out the buildworld and installworld.

How to configure mergemaster to merge configurations on FreeBSD

mariusvw — Sun, 10 Jul 2011 10:37:32 +0000

Ignore files for upgrade, for example you could add /etc/motd here.

Simply edit the file /etc/mergemaster.rc and add the following line:

IGNORE_FILES='/etc/motd'

Another example:

# Automatically install files that do not exist on the system already
AUTO_INSTALL=yes
 
# Automatically upgrade files that have not been user modified
AUTO_UPGRADE=yes
 
# Preserve files that you replace
PRESERVE_FILES=yes
PRESERVE_FILES_DIR=/var/tmp/mergemaster/preserved-files-`date +%y%m%d-%H%M%S`
 
# Files to always avoid comparing (space separated)
IGNORE_FILES='/etc/motd /etc/rc.conf'

For more information check out the manual page.

How to easy CVS update your SRC and Ports tree with a simple Bash script

mariusvw — Sun, 10 Jul 2011 10:34:24 +0000

Check this updated post: How to easy CVS update your SRC and Ports tree with a simple Bash script (v2)

Its advised to run this in Screen, you can install it from /usr/ports/sysutils/screen.

First install a simple tool to get the fastest cvs server

pkg_add -r fastest_cvsup

Or from Ports (if you have the collection already):

cd /usr/ports/sysutils/fastest_cvsup/
make install clean

Install CVSUP

pkg_add -r cvsup-without-gui

Or from Ports (if you have the collection already):

cd /usr/ports/net/cvsup-without-gui
make install clean

Setup config and binary to easy update the repository for SRC and PORTS.

mkdir /usr/local/etc/cvsup
cd /usr/share/examples/cvsup
cp stable-supfile /usr/local/etc/cvsup
cp ports-supfile /usr/local/etc/cvsup
cp doc-supfile /usr/local/etc/cvsup
cd /usr/local/etc/cvsup

Edit stable-supfile and change:

*default release=cvs tag=RELENG_7_1

Set this to your release version.

Edit /usr/local/etc/cvsup/runcvsup and add:

#!/bin/sh
arg=$@
sType="tld,nl" # Can be: all / tld / tld,nl,de,fr,us
if SERVER=`/usr/local/bin/fastest_cvsup -q -c $sType`
then
        echo "Using server: ${SERVER}"
        case "$arg" in
        "stable")
                echo "Updating: stable"
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/stable-supfile
        ;;
        "ports")
                echo "Updating: ports"
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/ports-supfile
        ;;
        "doc")
                echo "Updating: doc"
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/doc-supfile
        ;;
        *)
                echo "Updating: all"
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/stable-supfile
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/ports-supfile
                /usr/local/bin/cvsup -L 2 -g -z -h ${SERVER} /usr/local/etc/cvsup/doc-supfile
        ;;
        esac
fi

Set the right permissions:

chmod 700 runcvsup

Link the script so it can be reached.

ln -s /usr/local/etc/cvsup/runcvsup /usr/local/bin/runcvsup
rehash

Update everything:

runcvsup

How to use freebsd-update to update your FreeBSD kernel and world system

mariusvw — Sun, 10 Jul 2011 10:08:37 +0000

Information

http://www.freebsd.org/doc/en/books/handbook/updating-upgrading-freebsdupdate.html

Keep in mind, if you have a custom kernel that you need to upgrade your kernel manually!!!

Usage

The freebsd-update(8) utility supports binary upgrades of i386 and amd64 systems running earlier FreeBSD releases. Systems running 7.[01234]-RELEASE, 8.[01]-RELEASE, 8.2-BETA1, or 8.2-RC[123] can upgrade as follows:

# freebsd-update upgrade -r 8.2-RELEASE

During this process, FreeBSD Update may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly.

# freebsd-update install

The system must be rebooted with the newly installed kernel before continuing.

# shutdown -r now

After rebooting, freebsd-update needs to be run again to install the new userland components:

# freebsd-update install

At this point, users of systems being upgraded from FreeBSD 7.4-RELEASE or earlier will be prompted by freebsd-update to rebuild all third-party applications (e.g., ports installed from the ports tree) due to updates in system libraries.

After updating installed third-party applications (and again, only if freebsd-update printed a message indicating that this was necessary), run freebsd-update again so that it can delete the old (no longer used) system libraries:

# freebsd-update install

Finally, reboot into 8.2-RELEASE:

# shutdown -r now

How to install and use portaudit on FreeBSD

mariusvw — Sun, 10 Jul 2011 09:31:29 +0000

Installation

Install portaudit from the ports collection.

cd /usr/ports/ports-mgmt/portaudit
make install clean

Usage

When you install or upgrade a port, portaudit will automatically check for vurnability problems.

To disable the check while doing a //make install// add:

DISABLE_VULNERABILITIES=1

Manual check:

portaudit -Fda

Portupgrade
It might happen that you want to upgrade a port which has vulnerabilities. You need to pass on the make argument like this:

portupgrade -m DISABLE_VULNERABILITIES=1 'php52*'

How to configure Bind name server as caching LAN server on FreeBSD

mariusvw — Tue, 05 Jul 2011 20:04:22 +0000

On FreeBSD 8.x and higher this manual should work fine. Bind is default installed.

Edit /etc/namedb/named.conf and add/change the following.

options {} block

    listen-on {
        127.0.0.1;
        192.168.1.2;
    };
    forwarders {
        8.8.8.8;
        8.8.4.4;
    };

At the bottom of the file add your zones, example:

zone "kitara.nl" {
    type master;
    file "/etc/namedb/master/domain.tld-forward.db";
};

Edit your master file and add/edit this to your needs, open /etc/namedb/master/domain.tld-forward.db

$TTL 1h
kitara.nl. SOA kitara.nl. nobody.kitara.nl. 42 1d 12h 1w 3h
            ; Serial, Refresh, Retry, Expire, Neg. cache TTL
 
            NS  ns.kitara.nl.
 
            A   77.74.48.170
*           A   77.74.48.170
localhost   A   127.0.0.1
www         A   77.74.48.170
prn         A   172.16.32.12

Now edit /etc/rc.conf and add the startup value.

named_enable="YES"

Now start named

/etc/rc.d/named start

Now test your server by throwing in some queries:

# Short output
host google.com 127.0.0.1
# More output
dig @127.0.0.1 google.com

How to mount a Western Digital ShareSpace NFS share

mariusvw — Thu, 30 Jun 2011 18:42:59 +0000

This might become handy if you want to connect the drive on a FreeBSD or Linux machine to easy transfer files to it.

Enable NFS on your Western Digital ShareSpace

Log into your web interface and goto Advanced Mode

Goto the tab Network

Goto Services

Here check the Enable checkbox and add the IP of the machine you want to grant access

Next goto Storage

Edit the share preferences

Enable NFS support for this share

Goto Users

Edit the user preferences

Grand optional write access to this user

Connect to NFS mount of ShareSpace on FreeBSD

List mounts

showmount -e 192.168.2.3

Connect mountpoint

# Read only
mount -t nfs 192.168.2.3:/DataVolume/backup /mnt/
# Read / Write
mount -t nfs -o rw 192.168.2.3:/DataVolume/backup /mnt/

Add share to /etc/fstab

172.16.32.44:/DataVolume/backup /backup nfs rw  2   2

Mount with mount -a

How to backup to a Western Digital ShareSpace drive via SSH-only rsync from FreeBSD

mariusvw — Thu, 30 Jun 2011 17:39:22 +0000

This manual helps you to backup directories from your server to a Western Digital ShareSpace.

You might need this: How to enable SSH on a Western Digital ShareSpace

Create directories and generate a key for the target

mkdir /usr/data/backup
mkdir /usr/data/backup/keys
ssh-keygen -t rsa -f /usr/data/backup/keys/sharespace
find /usr/data/backup -type f -exec chmod 600 "{}" \; 
find /usr/data/backup -type d -exec chmod 700 "{}" \;

Add public key to the ShareSpace root account

ssh root@
 
cd /root
mkdir .ssh
chmod 700 .ssh
 
# Add key to this file.
vi ssh/authorized_keys
 
chmod 700 .ssh/authorized_keys

Add scripts to server

Place backup.sh in /usr/backup/ and functions.sh in /usr/backup/includes/. Then simply edit to your needs and run with ./backup.sh or from Cron.

backup.sh

#!/bin/bash
# Backup system
# by Marius van Witzenburg 
# http://kitara.nl
 
mailto="marius@kitara.nl"
mailfrom="noreply@kitara.nl"
 
root="/usr/data/backup"
 
###
# Do not edit below!
###
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
pid=$$
 
source $root/include/functions.sh
 
# Create needed directories
test -d $root/files || mkdir $root/files
test -d $root/include || mkdir $root/include
test -d $root/keys || mkdir $root/keys
test -d $root/log || mkdir $root/log
test -d $root/tmp || mkdir $root/tmp
test -d $root/tmp/server || mkdir $root/tmp/server
 
echo $pid > $root/tmp/backup.pid
 
# Sync Public
sync_public "192.168.1.50" "neo" "/usr/data/storage/public" 22
sync_public "192.168.2.50" "trinity" "/usr/data/storage/public" 12345
 
# Sync files to ShareSpace drives
sync_sharespace "192.168.1.50" "neo" "/usr/data/files" "files" 22
sync_sharespace "192.168.1.50" "neo" "/usr/data/music" "music" 22
sync_sharespace "192.168.1.50" "neo" "/usr/data/storage" "storage" 22
sync_sharespace "192.168.2.50" "trinity" "/usr/data/files" "files" 12345
sync_sharespace "192.168.2.50" "trinity" "/usr/data/music" "music" 12345
sync_sharespace "192.168.2.50" "trinity" "/usr/data/storage" "storage" 12345
 
# EOF

includes/functions.sh

#!/bin/bash
 
if [ -z "$root" ]
then
    echo "DO NOT CALL THIS FILE DIRECTLY!"
fi
 
date=`date "+%d-%m-%Y"`
date_reverse=`date "+%Y-%m-%d"`
hostname=`hostname -s`
hostletters=`hostname | sed -E 's/([A-Za-z0-9])[^.]*\.?/\1/g'`
sshport=22
 
# Usage: email "subject" "message"
email() {
    all=$@  
    subject=$1
    message=$2
    echo -e "From: $mailfrom\nTo: $mailto\nSubject: $subject\n\n$message" | /usr/sbin/sendmail -t $mailto 
}
 
# Cleanup loggin and temp directories
cleanup() {
    rm -r $root/log/
    rm -r $root/tmp/
}
 
stop_public() {
    all=$@  
    signal=$1
    name=$2 
    ip=$3   
 
    if [ "$signal" != "finish" ]
    then    
        if [ "$signal" == 'int' ] || [ "$signal" == 'term' ]
        then    
            cleanup 
        fi      
 
        logger -s -p daemon.notice -t 'backup' "stop_public(): Public Sync shutdown!. Target: $name ($ip), Signal: $signal"
        kill $JOB
 
        if [ "$signal" == 'int' ] || [ "$signal" == 'term' ]
        then    
            exit $? 
        else    
            return $?
        fi      
    fi
}
 
# Syntax: sync_public "srv_ip" "srv_name" "source"
sync_public() {
    all=$@  
    srv_ip=$1
    srv_name=$2
    src=$3  
    sshport=$4
 
    # Give it a rest so we don't flood the sshd!
    sleep 3 
 
    key="$root/keys/$srv_name"
 
    #trap "stop_public hub $srv_name $srv_ip" SIGHUP
    trap "stop_public int $srv_name $srv_ip" SIGINT
    trap "stop_public term $srv_name $srv_ip" SIGTERM 
 
    alive=`ssh -ax -q -q -o "BatchMode=yes" -o "ConnectTimeout 15" -i $key -p $sshport root@$srv_ip "echo 0 2>&1" && return 0 || echo 1`
    if [ $alive -eq 0 ] 
    then    
        rsync -e "ssh -ax -i $key -p $sshport" -aWvz --timeout=300 --delete-during $src/ root@$srv_ip:/shares/Public/ >> $root/log/run 2>&1 &
        JOB=$!  
        wait $JOB
 
        tail -2 $root/log/run | logger -p daemon.info -t 'backup'
 
        stop_public finish $srv_name $srv_ip
    else
        logger -s -p daemon.notice -t 'backup' "sync_public(): Connection to $srv_name ($srv_ip) failed..."
    fi
}  
 
stop_sharespace() {
    all=$@
    signal=$1
    name=$2
    ip=$3
    key=$4
 
    if [ "$signal" != "finish" ]
    then
        if [ "$signal" == 'int' ] || [ "$signal" == 'term' ]
        then
            cleanup
        fi
 
        logger -s -p daemon.notice -t 'backup' "stop_sharespace(): ShareSpace backup shutdown!. Target: $name ($ip), Signal: $signal"
        kill $JOB
 
        if [ "$signal" == 'int' ] || [ "$signal" == 'term' ]
        then
            exit $?
        else
            return $?
        fi
    fi
}  
 
sync_sharespace() {
    all=$@
    srv_ip=$1
    srv_name=$2
    src=$3
    target=$4
    sshport=$5
 
    # Give it a rest so we don't flood the sshd!
    sleep 3
 
    key="$root/keys/$srv_name"
 
    #trap "stop_sharespace hub $srv_name $srv_ip $key" SIGHUP
    trap "stop_sharespace int $srv_name $srv_ip $key" SIGINT
    trap "stop_sharespace term $srv_name $srv_ip $key" SIGTERM
 
    alive=`ssh -ax -q -q -o "BatchMode=yes" -o "ConnectTimeout 15" -i $key -p $sshport root@$srv_ip "echo 0 2>&1" && return 0 || echo 1`
    if [ $alive -eq 0 ]
    then
        # Create required directory for server name
        ssh -ax -i $key -p $sshport root@$srv_ip "test -d /shares/backup/$hostname || mkdir /shares/backup/$hostname"
        JOB=$!
        wait $JOB
 
        # Sync data
        logger -p daemon.notice -t 'backup' "sync_sharespace(): Syncing '$src'..."
        rsync -e "ssh -ax -i $key -p $sshport" -rtlDWvp --timeout=300 --chmod=Dug=rwX,Fug=rwX,Do=rX,Fo=r --delete-during $src/ root@$srv_ip:/shares/backup/$hostname/$target/ >> $root/log/run 2>&1 &
        JOB=$!
        wait $JOB
 
        tail -2 $root/log/run | logger -p daemon.info -t 'backup'
 
        stop_sharespace finish $srv_name $srv_ip $key
    else
        logger -s -p daemon.notice -t 'backup' "sync_sharespace(): Connection to $srv_name ($srv_ip) failed..."
    fi
}
 
# EOF

How to configure a automatically native IPv6 connection on FreeBSD

mariusvw — Sun, 26 Jun 2011 17:20:37 +0000

First you need to install a DHCP client

cd /usr/ports/net/dhcp6
make install clean

Setup config for dhcp6c

cp /usr/local/etc/dhcp6c.conf.sample /usr/local/etc/dhcp6c.conf
vi /usr/local/etc/dhcp6c.conf

Add the following

interface em0 {
    send ia-pd 0;
};
 
id-assoc pd {
    prefix-interface em0 {
        sla-id 1;
    };
};

Add the following to /etc/rc.conf

ipv6_enable="YES"
ipv6_network_interfaces="em0"
ipv6_defaultrouter="fe80::%em0"
dhcp6c_enable="YES"
dhcp6c_interfaces="em0"

Run the IPv6 startup script.

/etc/rc.d/network_ipv6 restart

Now startup the client

/usr/local/etc/rc.d/dhcp6c start

To test the connection you should be able to ping XS4ALL.

ping6 xs4all.nl

You also might want to check out these posts for some more information:
How to use a PPPoE or PPPoA connection of XS4ALL on FreeBSD

How to configure IPv6 DHCP client for XS4ALL on FreeBSD

How to configure a IPv6 gateway for XS4ALL on FreeBSD