wiki:NetPerfExample

Example of use on netperf (simple example)

As an example of how easy (or not :-) Project-Builder is to use, I decided to write that tutorial as a gift for another project which deserves it, the NetPerf project. This documentation corresponds to Project-Builder version 0.9.7.1.

You first have to create a .pbrc configuration file in your home directory, which will setup things globally:

$ cat > ~/.pbrc << 'EOF'
#
# Define for each project the URL of its pbconf repository
# No default option allowed here as they need to be all different
#
pbconfurl netperf = svn+ssh://svn.mondorescue.org/mondo/svn/project-builder/netperf/pbconf

#
# Under that dir will take place everything related to pb
# If you want to use VMs/chroot/..., then use $ENV{'HOME'} to make it portable
# to your VMs/chroot/...
# if not defined then /var/cache
pbdefdir default = $ENV{'HOME'}/project-builder

# location of Virtual Machine infrastructure
vmpath default = $ENV{'HOME'}/qemu

# location of Virtual Environment (chroot) infrastructure
vepath default = $ENV{'HOME'}/mock

EOF

Note that the format is in general

pb-keyword project = value

which means that here I named my project netperf for Project-Builder (I can choose what I want but need to be consistent everywhere). Also that configuration file is linked to the person who wants to package the software. Another configuration file will also be used, which only concerns the project itself.

The pbconfurl keyword indicates where are stored the configuration files for pb allowing us to build netperf packages with them. Here by default, everything will be extracted under $ENV{'HOME'}/project-builder for all projects we decide to package. Other keywords exist and may be used, but won't be needed for such a simple project.

To get the bits of the project you want to package, different protocols may be used. Currently supported protocols are using a SVN, SVK, CVS, Git or Mercurial access if there is such a thing, or a URL pointing to a ftp or http server.

The netperf project is a nice project using a simple environment to build itself. It uses the GNU autoconf/autmake/litool mechanism, which makes it easy to package with pb.

You will now need to get the latest and greatest project-builder software from our ftp repository. Of course we provide lots of packages for your distribution ;-) Or a tar.gz file if you prefer. You'll need at least version 0.9.0 of pb to support what is described here. Once available, you can use the newproj option of pb to create the infrastructure that is mandatory for pb. The latest version available of netperf at the time of this writing is 2.4.5. And the only package we want to create is netperf.

$ pb -p netperf -r 2.4.5 newproj netperf
Packages: netperf

Do not to forget to commit the pbconf directory in your CMS if needed
Project: netperf
Action: newproj

Explore then the pbconf directory to familiarize yourself with some of the config file. The next one we want to edit, is the main configuration file which will be used as the central piece of information for pb.

Edit it and adapt it. After edition, here is the resulting file:

$ cd $HOME/project-builder/
$ ls -R netperf/pbconf/2.4.5/
netperf/pbconf/2.4.5/:
netperf  netperf.pb  pbfilter

netperf/pbconf/2.4.5/netperf:
deb  pbfilter  rpm

netperf/pbconf/2.4.5/netperf/deb:
changelog  compat  control  copyright  netperf.docs  netperf.dirs  rules

netperf/pbconf/2.4.5/netperf/pbfilter:

netperf/pbconf/2.4.5/netperf/rpm:
netperf.spec

netperf/pbconf/2.4.5/pbfilter:
all.pbf  deb.pbf  md.pbf  novell.pbf  rpm.pbf
$
$ vi netperf/pbconf/2.4.5/netperf.pb
[Modifications here]
$ cat netperf/pbconf/2.4.5/netperf.pb
#
# Project Builder configuration file
# For project netperf
#
# $Id$
#

#
# What is the project URL
#
#pburl netperf = svn://svn.netperf.org/netperf/devel
#pburl netperf = svn://svn+ssh.netperf.org/netperf/devel
#pburl netperf = cvs://cvs.netperf.org/netperf/devel
#pburl netperf = http://www.netperf.org/src/netperf-devel.tar.gz
pburl netperf = ftp://ftp.netperf.org/netperf/netperf-2.4.5.tar.bz2
#pburl netperf = file:///src/netperf-devel.tar.gz
#pburl netperf = dir:///src/netperf-devel

# Check whether project is well formed 
# (containing already a directory with the project-version name)
pbwf netperf = 1

#
# Packager label
#
pbpackager netperf = Bruno Cornec <bruno@project-builder.org>
#
vmlist netperf = mandriva-2009.1-i386
pbrepo netperf = ftp://ftp.project-builder.org

#
# Global version/tag for the project
#
projver netperf = 2.4.5
projtag netperf = 1

# Hash of valid version names
version netperf = devel

# Adapt to your needs:
# Optional if you need to overwrite the global values above
#
#pkgver netperf = stable
#pkgtag netperf = 3
# Hash of default package/package directory
defpkgdir netperf = netperf-2.4.5
# Hash of additional package/package directory
#extpkgdir minor-pkg = dir-minor-pkg

# List of files per pkg on which to apply filters
# Files are mentioned relatively to pbroot/defpkgdir
#filteredfiles netperf = Makefile.PL,configure.in,install.sh,netperf.8
#supfiles netperf = netperf.init

I inform pb that I will produce packages with a version-tag of 2.4.5-1, that no local package should have a different version than the project version (pkgver/pkgtag), that my default package will be netperf (pb will work on it by default all the time) and that no additional packages will be produced. In addition to standard version with numbers, pb will accept the version devel. Once this is done, you may commit your changes in your repository and already try to see if pb can make something useful for you. Check the result of:

$ svn add netperf
A      netperf
A      netperf/pbconf
A      netperf/pbconf/2.4.5
A      netperf/pbconf/2.4.5/netperf.pb
[..]
A      netperf/pbconf/2.4.4/pbfilter/all.pbf
$ svn ci -m "Adding pb info for netperf"
Ajout          netperf
Ajout          netperf/pbconf
Ajout          netperf/pbconf/2.4.5
[...]
Ajout          netperf/pbconf/2.4.5/pbfilter/rpm.pbf
[...]
$ svn up
[...]
$ pb -p netperf -r 2.4.5 cms2build
Project: netperf
Action: cms2build
Packages: netperf

Management of netperf 2.4.5-1 (rev flat)
Downloading ftp://ftp.netperf.org/netperf/netperf-2.4.5.tar.bz2 with /usr/bin/wget -nv -O  to /home/bruno/tmp/pb.SEaHx3hxvu/netperf-2.4.5.tar.bz2
... OK
Extracting /home/bruno/tmp/pb.SEaHx3hxvu/netperf-2.4.5.tar.bz2 in /home/bruno/local/pb/projects/netperf/delivery ... OK
Build files generated for asianux-2-i386,asianux-2-x86_64,asianux-3-i386,asianux-3-x86_64,debian-3.1-i386,debian-4.0-i386,debian-4.0-x86_64,debian-5.0-i386,debian-5.0-x86_64,fedora-10-i386,fedora-10-x86_64,fedora-4-i386,fedora-5-i386,fedora-6-i386,fedora-6-x86_64,fedora-7-i386,fedora-7-x86_64,fedora-8-i386,fedora-8-x86_64,fedora-9-i386,fedora-9-x86_64,mandrake-10.1-i386,mandrake-10.2-i386,mandriva-2006.0-i386,mandriva-2007.0-i386,mandriva-2007.0-x86_64,mandriva-2007.1-i386,mandriva-2007.1-x86_64,mandriva-2008.0-i386,mandriva-2008.0-x86_64,mandriva-2008.1-i386,mandriva-2008.1-x86_64,mandriva-2009.0-i386,mandriva-2009.0-x86_64,mandriva-2009.1-i386,mandriva-2009.1-x86_64,redhat-7.3-i386,redhat-9-i386,rhel-2.1-i386,rhel-3-i386,rhel-4-i386,rhel-4-x86_64,rhel-5-i386,rhel-5-x86_64,sles-10-i386,sles-10-x86_64,sles-9-i386,suse-10.0-i386,suse-10.1-i386,suse-10.2-i386,suse-10.2-x86_64,suse-10.3-i386,suse-10.3-x86_64,suse-11.0-i386,suse-11.0-x86_64,suse-11.1-i386,suse-11.1-x86_64,ubuntu-6.06-i386,ubuntu-7.04-i386,ubuntu-7.04-x86_64,ubuntu-7.10-i386,ubuntu-7.10-x86_64,ubuntu-8.04-i386,ubuntu-8.04-x86_64,ubuntu-8.10-i386,ubuntu-8.10-x86_64,ubuntu-9.04-i386,ubuntu-9.04-x86_64
No Build files found for gentoo-nover-i386,gentoo-nover-x86_64
Delivering and compressing patches
Preserving original tar file Under /home/bruno/local/pb/projects/netperf/delivery/netperf-2.4.5.tar.gz
Creating pbconf tar files compressed... OK
Under /home/bruno/local/pb/projects/netperf/delivery/netperf-2.4.5.pbconf.tar.gz

Ok we tried to make a build file from our Configuration Management System (CMS) which worked completely out of the box as our structure is simple. In particular, pb already recognized the project name, action, package to build, type of CMS (flat), used our -r option to create the intermediate delivery directory where all the final files|packages will be created.

Now this is done, it's time to look at all those files that gave been generated in addition and adapt them. At the begining, we try to make RPM packages.

$ vi netperf/pbconf/2.4.5/netperf/rpm/netperf.spec
[Modifications here]
$ cat netperf/pbconf/2.4.5/netperf/rpm/netperf.spec
#
# $Id$
#

Summary:        The netperf tool is a benchmarking network tool
Summary(fr):    netperf est un outil de benchmark réseau

Name:           PBPKG
Version:        PBVER
Release:        PBTAGPBSUF
License:        PBLIC
Group:          PBGRP
Url:            PBURL
Source:         PBSRC
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(id -u -n)

%description
PBDESC

%description -l fr
netperf est un outil de benchmark réseau

%prep
%setup -q

%build
./configure
make %{?_smp_mflags} VERSION=%{version}

%install
%{__rm} -rf $RPM_BUILD_ROOT
make DESTDIR=$RPM_BUILD_ROOT install

%clean
%{__rm} -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root)
%doc ChangeLog
%doc INSTALL COPYING README AUTHORS NEWS

%changelog
PBLOG

That spec file looks normal except that it uses a lot of PBSOMETHING keywords. Those are Project-Builder macros that will be instantiated depending on the distribution during the cms2build step.

So now that we have the base for our spec file, we need to look at those filters:

$ vi netperf/pbconf/2.4.5/pbfilter/all.pbf
[Modifications here]
$ cat netperf/pbconf/2.4.5/pbfilter/all.pbf
#
# $Id$
#
# Filter for all files
#
# PBSRC is replaced by the source package format
filter PBSRC = ftp://ftp.netperf.org/netperf/%{name}-%{version}.tar.gz

# PBVER is replaced by the version ($pb->{'ver'} in code)
filter PBVER = $pb->{'ver'}

# PBDATE is replaced by the date ($pb->{'date'} in code)
filter PBDATE = $pb->{'date'}

# PBLOG is replaced by the changelog if value is yes
filter PBLOG = yes

# PBTAG is replaced by the tag ($pb->{'tag'} in code)
filter PBTAG = $pb->{'tag'}

# PBREV is replaced by the revision ($pb->{'rev'} in code)
filter PBREV = $pb->{'rev'}

# PBPKG is replaced by the package name ($pb->{'pkg'} in code)
filter PBPKG = $pb->{'pkg'}

# PBPACKAGER is replaced by the packager name ($pb->{'packager'} in code)
filter PBPACKAGER = $pb->{'packager'}

# PBDESC contains the description of the package
filter PBDESC = "netperf is a network benchmarking tool"

# PBURL contains the URL of the Web site of the project
filter PBURL = http://www.netperf.org

We can now try again to use pb to create again our build file with macro expansion:

$ pb -p netperf -r 2.4.5 cms2build
Project: netperf
Action: cms2build
Packages: netperf

Management of netperf 2.4.5-1 (rev flat)
[... same output as previously...]
Under /home/bruno/project-builder/netperf/delivery/netperf-2.4.5.pbconf.tar.gz

You have now your first useful result from pb :-) Congrats ! Well wait ! It's not really useful because we started exactly from a tar file. Well now we have 2 ! This last one adds Project-Builder support:

$ tar tvfz /home/bruno/project-builder/netperf/delivery/netperf-2.4.5.pbconf.tar.gz
drwxr-xr-x bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/                                                                          
drwxr-xr-x bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/                                                        
-rw-r--r-- bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/netperf.dirs                                            
-rw-r--r-- bruno/bruno       2 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/compat                                                  
-rw-r--r-- bruno/bruno    3945 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/changelog                                               
-rw-r--r-- bruno/bruno    3632 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/rules                                                   
-rw-r--r-- bruno/bruno      36 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/netperf.docs                                            
-rw-r--r-- bruno/bruno     433 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/control                                                 
-rw-r--r-- bruno/bruno     994 2009-07-12 14:20 netperf-2.4.5/pbconf/debian-4.0-x86_64/copyright                                               
drwxr-xr-x bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/mandriva-2007.1-i386/                                                     
-rw-r--r-- bruno/bruno    4614 2009-07-12 14:20 netperf-2.4.5/pbconf/mandriva-2007.1-i386/netperf.spec                                         
[...]
drwxr-xr-x bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/
-rw-r--r-- bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/netperf.dirs
-rw-r--r-- bruno/bruno       2 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/compat
-rw-r--r-- bruno/bruno    3945 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/changelog
-rw-r--r-- bruno/bruno    3632 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/rules
-rw-r--r-- bruno/bruno      36 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/netperf.docs
-rw-r--r-- bruno/bruno     433 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/control
-rw-r--r-- bruno/bruno     994 2009-07-12 14:20 netperf-2.4.5/pbconf/ubuntu-8.04-i386/copyright
drwxr-xr-x bruno/bruno       0 2009-07-12 14:20 netperf-2.4.5/pbconf/fedora-5-i386/
-rw-r--r-- bruno/bruno    4608 2009-07-12 14:20 netperf-2.4.5/pbconf/fedora-5-i386/netperf.spec

As you can see a lot of directories and files have been created under pbconf. They correspond to all the required build files for all the distributions supported and will later be used to really create the packages.

So now, we can try to go to the next step and build a package. So issue:

$ pb -p netperf -r 2.4.5 build2pkg
Project: netperf
Action: build2pkg
Packages: netperf
Extracting spec files from /home/bruno/local/pb/projects/netperf/delivery/netperf-2.4.5.pbconf.tar.gz... OK                                    
Extracting patch files from /home/bruno/local/pb/projects/netperf/delivery/netperf-2.4.5.pbconf.tar.gz... N/A                                  
Building package with /home/bruno/local/pb/projects/netperf/build/SPECS/netperf.spec under /home/bruno/local/pb/projects/netperf/build... child (rpmbuild  --define 'packager Bruno Cornec <bruno@project-builder.org>' --define "_topdir /home/bruno/local/pb/projects/netperf/build" -ba /home/bruno/local/pb/projects/netperf/build/SPECS/netperf.spec) exited with value 1                       
Executing(%prep): /bin/sh -e /home/bruno/project-builder/netperf/build/tmp/rpm-tmp.70387
+ umask 022
+ cd /home/bruno/project-builder/netperf/build/BUILD
+ '[' 1 -eq 1 ']'
+ '[' 1 -eq 1 ']'
+ '[' 1 -eq 1 ']'
+ cd /home/bruno/project-builder/netperf/build/BUILD
+ rm -rf netperf-2.4.5
+ /usr/bin/gzip -dc /home/bruno/project-builder/netperf/build/SOURCES/netperf-2.4.5.tar.gz
+ tar -xf -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd netperf-2.4.5
+ exit 0
Executing(%build): /bin/sh -e /home/bruno/project-builder/netperf/build/tmp/rpm-tmp.70387
+ umask 022
+ cd /home/bruno/project-builder/netperf/build/BUILD
+ cd netperf-2.4.5
+ '[' 1 -eq 1 ']'
+ '[' 1 -eq 1 ']'
+ CONFIGURE_TOP=.
+ CFLAGS='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fexceptions'
+ export CFLAGS
+ CXXFLAGS='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fexceptions'
+ export CXXFLAGS
+ FFLAGS='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fexceptions'
+ export FFLAGS
+ cputoolize -c .
+ '[' -f ./configure.in -o -f ./configure.ac ']'
+ CONFIGURE_XPATH='--x-includes=/usr/include --x-libraries=/usr/lib64'
+ ./configure x86_64-mandriva-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/lib64 --localstatedir=/var/lib --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --x-includes=/usr/include --x-libraries=/usr/lib64
configure: WARNING: you should use --build, --host, --target
checking build system type... x86_64-mandriva-linux-gnu
checking host system type... x86_64-mandriva-linux-gnu
[...]
config.status: creating netperf.spec
config.status: creating config.h
config.status: executing depfiles commands
+ make -j4 VERSION=2.4.5
make  all-recursive
make[1]: Entering directory `/home/bruno/project-builder/netperf/build/BUILD/netperf-2.4.5'
Making all in src
make[2]: Entering directory `/home/bruno/project-builder/netperf/build/BUILD/netperf-2.4.5/src'
[...]
if x86_64-mandriva-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I. -I..     -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fexceptions -MT netperf.o -MD -MP -MF ".deps/netperf.Tpo" \
          -c -o netperf.o `test -f 'netperf.c' || echo './'`netperf.c; \
        then mv -f ".deps/netperf.Tpo" ".deps/netperf.Po"; \
        else rm -f ".deps/netperf.Tpo"; exit 1; \
        fi
[...]
Checking for unpackaged file(s): /usr/lib/rpm/check-files /users/bruno/project-builder/netperf/build/tmp/netperf-2.4.5-1PBSUF-buildroot
error: Installed (but unpackaged) file(s) found:
   /usr/bin/netperf
   /usr/bin/netserver
   /usr/share/info/netperf.info.lzma
   /usr/share/man/man1/netperf.1.lzma
   /usr/share/man/man1/netserver.1.lzma


RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/bin/netperf
   /usr/bin/netserver
   /usr/share/info/netperf.info.lzma
   /usr/share/man/man1/netperf.1.lzma
   /usr/share/man/man1/netserver.1.lzma
child (rpmbuild --define 'packager "Bruno Cornec <bruno@project-builder.org>"' --define "_topdir /home/bruno/project-builder/netperf/build" -ba /home/bruno/project-builder/netperf/build/SPECS/netperf.spec) exited with value 1
Checking validity of rpms with rpmlint... Error: no installed packages by name RPMS/*/netperf-2.4.5-1.mdv2009.1.*.rpm
Error: no installed packages by name SRPMS/netperf-2.4.5-1.mdv2009.1.src.rpm
OK

So it seems we are not there yet ;-) But it's not too bad for just having edited a couple of files and parameters in our conf files. Now we need to solve those issues and produce the package.

First it seems that the PBSUF macro isn't expanded correctly. Look for it using a Project-Builder tool:

$ (cd netperf/pbconf ; pbg PBSUF)
./2.4.5/pbfilter/rpm.pbf:# PBSUF is replaced by the package name ($pbpkg in code)
./2.4.5/pbfilter/rpm.pbf:#filter PBSUF = $pb->{'suf'}
./2.4.5/netperf/rpm/netperf.spec:Release:        PBTAGPBSUF

So indeed that macro is used in our spec file, but is only declared for rpm types of build in the rpm filter file. However it's currently commented. So edit that file to validate it, as well as the other variables needed:

$ cat netperf/pbconf/2.4.5/pbfilter/rpm.pbf
#
# $Id$
#
# Filter for rpm build
#

# PBGRP is replaced by the RPM group of apps
# Cf: http://fedoraproject.org/wiki/RPMGroups
#filter PBGRP = Applications/Archiving

# PBLIC is replaced by the license of the application
# Cf: http://fedoraproject.org/wiki/Licensing
#filter PBLIC = GPL

# PBDEP is replaced by the list of dependencies
#filter PBDEP =

# PBSUF is replaced by the package suffix ($pbsuf in code)
filter PBSUF = $pb->{'suf'}

# PBOBS is replaced by the Obsolete line
#filter PBOBS =

And our final current problem is the list of files produced by the build process that need to be referenced in our %files session. So modify that section of the spec file to read like this:

%files
%defattr(-,root,root)
%doc ChangeLog
%doc INSTALL COPYING README AUTHORS NEWS
%doc README.* Release_Notes doc/examples
%{_bindir}/*
%{_mandir}/*/*
%{_infodir}/*

Now you're ready for your second run ! This time chain the 2 steps by generating from CMS to a package with cms2pkg (as you need to reparse the conf files):

$ pb -p netperf -r 2.4.5 cms2pkg
[... build process ...]
Wrote: /home/bruno/project-builder/netperf/build/SRPMS/netperf-2.4.5-1.mdv2009.1.src.rpm
Wrote: /home/bruno/project-builder/netperf/build/RPMS/x86_64/netperf-2.4.5-1.mdv2009.1.x86_64.rpm
Wrote: /home/bruno/project-builder/netperf/build/RPMS/x86_64/netperf-debug-2.4.5-1.mdv2009.1.x86_64.rpm
[... build process ...]
Checking validity of rpms with rpmlint... W: netperf manpage-not-bzipped /usr/share/man/man1/netperf.1.lzma
W: netperf infopage-not-bzipped /usr/share/info/netperf.info.lzma
W: netperf manpage-not-bzipped /usr/share/man/man1/netserver.1.lzma
E: netperf zero-length /usr/share/doc/netperf/NEWS
E: netperf info-files-without-install-info-postin /usr/share/info/netperf.info.lzma
[...]
$

So this time we have an RPM ! You can try to install it and check if it works:

$ sudo urpmi /users/bruno/project-builder/netperf/build/RPMS/x86_64/netperf-2.4.4-1.mdv2008.0.x86_64.rpm 

installation de netperf-2.4.4-1.mdv2008.0.x86_64.rpm depuis /users/bruno/project-builder/netperf/build/RPMS/x86_64
Préparation ...                  ##########################################################################
      1/1: netperf               ##########################################################################

$ sudo netserver
Starting netserver at port 12865
Starting netserver at hostname 0.0.0.0 port 12865 and family AF_UNSPEC
$ sudo netperf
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to victoria2.home.musique-ancienne.org (127.0.0.1) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.00    8697.72

So it seems to work. Check now the RPM:

$ rpm -qlv netperf
-rwxr-xr-x    1 root    root           103160 mar 31 00:06 /usr/bin/netperf
-rwxr-xr-x    1 root    root           110904 mar 31 00:06 /usr/bin/netserver
drwxr-xr-x    2 root    root                0 mar 31 00:06 /usr/share/doc/netperf
-rw-r--r--    1 root    root             6013 oct 15 19:41 /usr/share/doc/netperf/AUTHORS
-rw-r--r--    1 root    root             2079 jun  2  2007 /usr/share/doc/netperf/COPYING
-rw-r--r--    1 root    root               27 jun  2  2007 /usr/share/doc/netperf/ChangeLog
-rw-r--r--    1 root    root             9240 jun  2  2007 /usr/share/doc/netperf/INSTALL
-rw-r--r--    1 root    root                0 jun  2  2007 /usr/share/doc/netperf/NEWS
[...]
-rw-r--r--    1 root    root            32819 oct 17 23:20 /usr/share/doc/netperf/Release_Notes
drwxr-xr-x    2 root    root                0 mar 31 00:06 /usr/share/doc/netperf/examples
-rw-r--r--    1 root    root             7986 mar 31 00:06 /usr/share/doc/netperf/examples/Makefile
-rw-r--r--    1 root    root              467 jun  2  2007 /usr/share/doc/netperf/examples/Makefile.am
[...]
-rw-r--r--    1 root    root            30803 mar 31 00:06 /usr/share/info/netperf.info.lzma
-rw-r--r--    1 root    root             2702 mar 31 00:06 /usr/share/man/man1/netperf.1.lzma
-rw-r--r--    1 root    root              696 mar 31 00:06 /usr/share/man/man1/netserver.1.lzma
$ rpm -qi netperfName        : netperf                      Relocations: (not relocatable)
Version     : 2.4.5                             Vendor: (none)
Release     : 1mdv2009.1                    Build Date: dim. 12 juil. 2009 11:26:24 CEST
Install Date: dim. 12 juil. 2009 11:26:42 CEST      Build Host: localhost
Group       : Networking/Other              Source RPM: netperf-2.4.5-1mdv2009.1.src.rpm
Size        : 432811                           License: BSD
Signature   : (none)
URL         : http://www.netperf.org/netperf/NetperfPage.html
Summary     : Performance testing tool for TCP/UDP
Description :
Netperf is a benchmark that can be used to measure the performance
of many different types of networking. It provides tests for both
unidirecitonal throughput, and end-to-end latency.

The environments currently measureable by netperf include:

* TCP and UDP via BSD Sockets for both IPv4 and IPv6
* DLPI
* Unix Domain Sockets
* SCTP for both IPv4 and IPv6
$ rpm -q --changelog netperf
$ 

So we still have an issue with the changelog which is empty. Also some modes are wrong and our NEWS file is empty. Again Project-Builder can deal with those issues. You'll have for that to create a new file under pbconf/netperf called pbcl which will be the changelog file for your project:

$ cp netperf/build/BUILD/netperf-2.4.5/Release_Notes netperf/pbconf/2.4.5/netperf/pbcl
$ vi netperf/pbconf/2.4.5/netperf/pbcl
[... format modifications ...]
$ head -20 netperf/pbconf/2.4.5/netperf/pbcl
# $Id$

NETPERF CHANGES

2.4.5 (2009-07-12)
-  Update to upstream 2.4.5

The format of the pbcl file is strict. It has 4 initital lines that you can use the way you want, the 5th line should have 'Version (date)' with date using ISO format, then the following lines are the changelog content of that version. Then after a blank line, again a 'Version (date)' and the changelog, etc...

Ready to check again:

$ pb -p netperf -r 2.4.5 cms2pkg
[... build continues ...]

More verifications:

$ sudo rpm -Uvh --force /home/bruno/project-builder/netperf/build/RPMS/x86_64/netperf-2.4.5-1.mdv2009.1.x86_64.rpm

Préparation...              ########################################### [100%]
   1:netperf                ########################################### [100%]
$ rpm -qp --changelog /home/bruno/project-builder/netperf/build/RPMS/i586/netperf-2.4.5-1.mdv2009.1.x86_64.rpm | head
* dim. juil. 12 2009 Bruno Cornec <bruno@project-builder.org> 2.4.5-1
- Updated to 2.4.5
- Update to upstream 2.4.5

* dim. mars 30 2008 Bruno Cornec <bruno@project-builder.org> 2.4.4-1
- Updated to 2.4.4
-  The LOC_CPU and REM_CPU tests will report their respective beliefs
   as to the number of CPUs present when the verbosity is set to more
   than one.  This can be used when trying to diagnose issues with CPU
   utilization.
$ 

So we now have a much more correct rpm built, even if not everything is solved (In order to get a good NEWS file, we should remove the empty one part of netperf). The beauty of Project-Builder will now appear, as it's as easy to generate more RPMs for other distributions than it is for your native distribution.

Imagine you want to build the same RPMs for Fedora 6, instead of your native distribution. First download the ISO image of this distribution and put it where you have space on your disk. Then create a new configuration file for your build environment:

cat > /users/qemu/.pbrc << 'EOF'
#                                                               
# Valid values for vmtype are                                   
# qemu, kvm, (xen, ... TBD)                                     
vmtype default =qemu                                            

# Hash for VM stuff on vmtype
vmntp default = ntp.home.musique-ancienne.org
vmntpcmd default = /usr/sbin/ntpdate
vmntpcmd suse-11.0-i386 = sntp -P no -r
vmntpcmd suse-11.0-x86_64 = sntp -P no -r

# We suppose we can commmunicate with the VM through SSH
vmhost default = localhost
vmlogin default = pb
vmport default = 2222
vmmonport default = 4444

# Timeout to wait when VM is launched/stopped
vmtmout default = 180
vmtmout suse-10.2-i386 = 500
vmtmout mandriva-2009.0-i386 = 500
vmtmout mandriva-2009.0-x86_64 = 500

# per VMs needed paramaters
vmopt default = -m 384 -daemonize
vmopt suse-10.2-i386 = -m 384 -daemonize -no-kvm
vmopt suse-10.2-x86_64 = -m 384 -daemonize -no-kvm
vmopt mandriva-2009.0-i386 = -m 384 -daemonize -no-kvm
vmopt mandriva-2009.0-x86_64 = -m 384 -daemonize -no-kvm
#-no-kvm
#vmopt suse-10.2-i386 = -m 384 -daemonize -no-kqemu
#vmopt rhel-2.1-i386 = -m 384 -daemonize -no-kqemu
#vmopt mandriva-2007.0-x86_64 = -m 384 -daemonize -no-kqemu
vmsize default = 7G

# Do we use snapshot mode for VMs ?
# In which case all modifs in the VM are lost at the end
#vmsnap default = true

#
# For Virtual machines management
# Naming convention to follow: distribution name (as per ProjectBuilder::Distribution)
# followed by '-' and by release number
# followed by '-' and by architecture
# a .qemu extension will be added to the resulting string for those vmtype using that format
# a QEMU rhel-3-i386 here means that the VM will be named rhel-3-i386.qemu
# a XEN rhel-5-x86_64 here means that the VM will be named rhel-5-x86_64.qemu
#
vmlist default = asianux-2-i386,asianux-3-i386,mandrake-10.1-i386,mandrake-10.2-i386,mandriva-2006.0-i386,mandriva-2007.0-i386,mandriva-2007.1-i386,mandriva-2008.0-i386,mandriva-2008.1-i386,mandriva-2009.0-i386,mandriva-2009.1-i386,redhat-7.3-i386,redhat-9-i386,fedora-4-i386,fedora-5-i386,fedora-6-i386,fedora-7-i386,fedora-8-i386,fedora-9-i386,fedora-10-i386,rhel-2.1-i386,rhel-3-i386,rhel-4-i386,rhel-5-i386,suse-10.0-i386,suse-10.1-i386,suse-10.2-i386,suse-10.3-i386,suse-11.0-i386,suse-11.1-i386,sles-9-i386,sles-10-i386,gentoo-nover-i386,debian-3.1-i386,debian-4.0-i386,debian-5.0-i386,ubuntu-6.06-i386,ubuntu-7.04-i386,ubuntu-7.10-i386,ubuntu-8.04-i386,ubuntu-8.10-i386,,ubuntu-9.04-i386,asianux-2-x86_64,asianux-3-x86_64,mandriva-2007.0-x86_64,mandriva-2007.1-x86_64,mandriva-2008.0-x86_64,mandriva-2008.1-x86_64,mandriva-2009.0-x86_64,mandriva-2009.1-x86_64,fedora-6-x86_64,fedora-7-x86_64,fedora-8-x86_64,fedora-9-x86_64,fedora-10-x86_64,rhel-4-x86_64,rhel-5-x86_64,suse-10.2-x86_64,suse-10.3-x86_64,suse-11.0-x86_64,suse-11.1-x86_64,sles-10-x86_64,gentoo-nover-x86_64,debian-4.0-x86_64,debian-5.0-x86_64,ubuntu-7.04-x86_64,ubuntu-7.10-x86_64,ubuntu-8.04-x86_64,ubuntu-8.10-x86_64,ubuntu-9.04-x86_64
EOF

It contains all the info around your QEMU/KVM usage, from the type of virtualisation (qemu here), to the list of VMs we need to support (you need now to remove the vmlist entry from the netperf.pb conf file BTW), and some specificities per VM if needed.

Then call pb to help you create a new virtual machine for that distribution:

$ pb -p netperf -m fedora-6-i386 -i ~/Download/Fedora-6-i386-DVD.iso newvm
Project: nerperf
Action: newvm
Creating the QEMU VM... Formating '/users/qemu/fedora-6-i386.qemu', fmt=qcow2, size=5242880 kB
OK
Launching the VM /users/qemu/fedora-6-i386.qemu... OK
Waiting for VM fedora-6-i386 to come up...

Then the Virtual Machine is launched booting on your CD, and the rest is just a classical installation of a Linux distribution. Just check that you've installed a sshd package (generally OpenSSH) and that you're allowed to login as root via ssh. This will be used in the next step. I also especially disable firewall and SElinux on the VM (you don't need security in that private build environement). Once your distribution is installed, stop the VM and relaunch it to finish the setup with:

$ pb -p netperf -m fedora-6-i386 launchvm
Project: netperf
Action: launchvm
Launching the VM /users/qemu/fedora-6-i386.qemu... OK
Waiting for VM fedora-6-i386 to come up...           

At the end of this initial setup phase log as root on your new VM installation. Check that your update mechanism for yum is correct and that you can connect on this VM as root with ssh.

So now you can finish to set up the VM by installing and configuring all what is necessary for pb to work flawlessly. There is an option to pb to exactly do that:

$ pb -p netperf -m fedora-6-i386 setupvm
Project: netperf
Action: setupvm
Copying local keys to vm. This will require the root password... root@localhost passwd:
Found an existing VM /users/qemu/fedora-6-i386.qemu (pid 30675)
Packages: netperf
Sources handled (Script): /users/bruno/project-builder/netperf/delivery/pbscript
Preparing project-builder/src on root@localhost:project-builder/src... OK
pbscript                                                                   100%   12KB  11.7KB/s   00:00
OK
Executing pbscript on root@localhost:project-builder/src if needed... distro tuple: fedora,6,rh,rpm,.fc6
Loading "installonlyn" plugin
Cleaning up Everything
Loading "installonlyn" plugin
[...]

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 perl-DateManip          noarch     5.44-2.fc6       updates           143 k
 rpm-build               i386       4.4.2.1-2.fc6    updates           640 k
Updating:
 ntp                     i386       4.2.4p2-1.fc6    updates           1.4 M
 wget                    i386       1.10.2-8.fc6.1   updates           581 k
Installing for dependencies:
 elfutils                i386       0.131-1.fc6      updates           211 k
 elfutils-libelf-devel-static  i386       0.131-1.fc6      updates            62 k
Updating for dependencies:
 elfutils-libelf         i386       0.131-1.fc6      updates            56 k
 elfutils-libelf-devel   i386       0.131-1.fc6      updates            23 k
 elfutils-libs           i386       0.131-1.fc6      updates           120 k
 popt                    i386       1.10.2.1-2.fc6   updates            70 k
 rpm                     i386       4.4.2.1-2.fc6    updates           1.1 M
 rpm-devel               i386       4.4.2.1-2.fc6    updates           5.2 M
 rpm-libs                i386       4.4.2.1-2.fc6    updates           921 k
 rpm-python              i386       4.4.2.1-2.fc6    updates            57 k

Transaction Summary
=============================================================================
Install      4 Package(s)
Update      10 Package(s)
Remove       0 Package(s)

Total download size: 11 M
Downloading Packages:
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating  : elfutils-libelf              ####################### [ 1/24]
  Updating  : popt                         ####################### [ 2/24]
  Updating  : elfutils-libs                ####################### [ 3/24]
  Installing: elfutils                     ####################### [ 4/24]
  Updating  : elfutils-libelf-devel        ####################### [ 5/24]
  Installing: perl-DateManip               ####################### [ 6/24]
  Updating  : ntp                          ####################### [ 7/24]
  Updating  : wget                         ####################### [ 8/24]
  Updating  : rpm-libs                     ####################### [ 9/24]
  Updating  : rpm                          ####################### [10/24]
  Updating  : rpm-python                   ####################### [11/24]
  Updating  : rpm-devel                    ####################### [12/24]
  Installing: rpm-build                    ####################### [13/24]
  Installing: elfutils-libelf-devel-static ####################### [14/24]

Installed: perl-DateManip.noarch 0:5.44-2.fc6 rpm-build.i386 0:4.4.2.1-2.fc6
Dependency Installed: elfutils.i386 0:0.131-1.fc6 elfutils-libelf-devel-static.i386 0:0.131-1.fc6
Updated: ntp.i386 0:4.2.4p2-1.fc6 wget.i386 0:1.10.2-8.fc6.1
Dependency Updated: elfutils-libelf.i386 0:0.131-1.fc6 elfutils-libelf-devel.i386 0:0.131-1.fc6 elfutils-libs.i386 0:0.131-1.fc6 popt.i386 0:1.10.2.1-2.fc6 rpm.i386 0:4.4.2.1-2.fc6 rpm-devel.i386 0:4.4.2.1-2.fc6 rpm-libs.i386 0:4.4.2.1-2.fc6 rpm-python.i386 0:4.4.2.1-2.fc6
Complete!
[...]
01:46:25 (133 KB/s) - `project-builder-latest.tar.gz' saved [62364]

project-builder-devel/
project-builder-devel/pbconf/
project-builder-devel/pbconf/suse-10.0/
project-builder-devel/pbconf/suse-10.0/project-builder.spec
project-builder-devel/pbconf/rhel-5/
[...]
project-builder-devel/contrib/newpbqemu
project-builder-devel/contrib/pbsetupqemu
project-builder-devel/INSTALL
Writing Makefile for pb
cp lib/ProjectBuilder/Base.pm blib/lib/ProjectBuilder/Base.pm
cp lib/ProjectBuilder/Distribution.pm blib/lib/ProjectBuilder/Distribution.pm
cp lib/ProjectBuilder/Version.pm blib/lib/ProjectBuilder/Version.pm
cp bin/pbvi blib/bin/pbvi
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/bin/pbvi
cp bin/pb blib/bin/pb
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/bin/pb
cp bin/pbdistrocheck blib/bin/pbdistrocheck
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/bin/pbdistrocheck
cp bin/pbg blib/bin/pbg
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/bin/pbg
Manifying blib/man1/pb.1
Writing /usr/lib/perl5/5.8.8/i386-linux-thread-multi/auto/pb/.packlist
Appending installation info to /usr/lib/perl5/5.8.8/i386-linux-thread-multi/perllocal.pod
OK
$ pb | head 
pb (aka project-builder.org) Version 0.9.7.1-818

Usage:
    pb [-vhq][-r pbroot][-p project][[-s script -a account -P port][-m
    mach-1[,...]]][-i iso] <action> [<pkg1> ...]

    pb [--verbose][--help][--man][--quiet][--revision pbroot][--project
    project][[--script script --account account --port port][--machine
    mach-1[,...]]][--iso iso] <action> [<pkg1> ...]

Now that Project-Builder is installed inside your VM, you may want to try to use it to build netperf packages for Fedora 6:

$ pb -p netperf -r 2.4.5 -m fedora-6-i386 build2vm
Project: netperf
Action: build2vm
Packages: netperf
Found an existing VM /users/qemu/fedora-6-i386.qemu (pid 31466)
Packages: netperf
Sources handled (vm): /home/bruno/project-builder/netperf/delivery/netperf-2.4.5.tar.gz /home/bruno/project-builder/netperf/delivery/pbscript /home/bruno/project-builder/netperf/pbconf/2.4.4/netperf.pb /home/bruno/project-builder/netperf/delivery/2.4.5-1.pb /home/bruno/.pbrc /home/bruno/project-builder/netperf/delivery/pbrc
Preparing project-builder/netperf/delivery on pb@localhost:project-builder/netperf/delivery... OK
VMbuild delivery in pb@localhost:local/pb/projects/netperf/delivery... OK                                                                                       
Executing pbscript on pb@localhost:local/pb/projects/netperf/delivery if needed... OK                                                                           
... Execution needed                                                                                                                                            
setting up date with echo sudo /usr/sbin/ntpdate ntp.home.musique-ancienne.org                                                                                  
[...]
building packages on vm...                                    
Project: netperf                                              
Action: build2pkg                                             
Packages: netperf                                             
Extracting spec files from /home/pb/local/pb/projects/netperf/delivery/netperf-2.4.5.pbconf.tar.gz... OK
Extracting patch files from /home/pb/local/pb/projects/netperf/delivery/netperf-2.4.5.pbconf.tar.gz... OK
Building package with /home/pb/local/pb/projects/netperf/build/SPECS/netperf.spec under /home/pb/local/pb/projects/netperf/build... OK
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.87737                                                                                   
+ umask 022
+ cd /home/pb/project-builder/netperf/build/BUILD
[... build goes on ...]Wrote: /home/pb/local/pb/projects/netperf/build/SRPMS/netperf-2.4.5-1.fc6.src.rpm
Wrote: /home/pb/local/pb/projects/netperf/build/RPMS/i386/netperf-2.4.5-1.fc6.i386.rpm
Wrote: /home/pb/local/pb/projects/netperf/build/RPMS/i386/netperf-debuginfo-2.4.5-1.fc6.i386.rpm
Checking validity of rpms with rpmlint... OK
[... build goes on ...]Get package names in pb@localhost:local/pb/projects/netperf/build... OK
Package recovery of netperf-2.4.5-1.fc6.src.rpm in pb@localhost:local/pb/projects/netperf/build... OK
Package recovery of netperf-2.4.5-1.fc6.i386.rpm in pb@localhost:local/pb/projects/netperf/build... OK
Package recovery of netperf-debuginfo-2.4.5-1.fc6.i386.rpm in pb@localhost:local/pb/projects/netperf/build... OK
VMbuild cleanup... OK
Packages: netperf
Sources handled (Packages): fedora/6/netperf-2.4.5-1.fc6.src.rpm fedora/6/netperf-2.4.5-1.fc6.i386.rpm fedora/6/netperf-debuginfo-2.4.5-1.fc6.i386.rpm /home/bruno/local/pb/projects/netperf/build/pbscript
Preparing /mondo/ftp/fedora/6 on pb@www.project-builder.org:/mondo/ftp/fedora/6... OK
netperf-2.4.5-1.fc6.i386.rpm                                               100%  163KB 163.2KB/s   00:00
netperf-2.4.5-1.fc6.src.rpm                                                100% 1156KB  31.2KB/s   00:37
OK
Executing pbscript on pb@www.project-builder.org:/mondo/ftp/fedora/6 if needed... OK
VM fedora-6-i386 halt (pid 0)... OK

We've been able to build our packages on Fedora 6 and at the end, as the build was successful they were automatically copied on our ftp server for public availability. In order to continue to support correctly netperf build, now that we have rpm done, we can attack the deb part. We need again to update the templates created by Project-Builder earlier: We can now try to generate packages for all the distributions supported:

$ pb -p netperf build2vm
Project: netperf
Action: build2vm
VMs: asianux-2-i386,asianux-3-i386,mandrake-10.1-i386,mandrake-10.2-i386,mandriva-2006.0-i386,mandriva-2007.0-i386,mandriva-2007.1-i386,mandriva-2008.0-i386,mandriva-2008.1-i386,mandriva-2009.0-i386,mandriva-2009.1-i386,redhat-7.3-i386,redhat-9-i386,fedora-4-i386,fedora-5-i386,fedora-6-i386,fedora-7-i386,fedora-8-i386,fedora-9-i386,fedora-10-i386,rhel-2.1-i386,rhel-3-i386,rhel-4-i386,rhel-5-i386,suse-10.0-i386,suse-10.1-i386,suse-10.2-i386,suse-10.3-i386,suse-11.0-i386,suse-11.1-i386,sles-9-i386,sles-10-i386,gentoo-nover-i386,debian-3.1-i386,debian-4.0-i386,debian-5.0-i386,ubuntu-6.06-i386,ubuntu-7.04-i386,ubuntu-7.10-i386,ubuntu-8.04-i386,ubuntu-8.10-i386,,ubuntu-9.04-i386,asianux-2-x86_64,asianux-3-x86_64,mandriva-2007.0-x86_64,mandriva-2007.1-x86_64,mandriva-2008.0-x86_64,mandriva-2008.1-x86_64,mandriva-2009.0-x86_64,mandriva-2009.1-x86_64,fedora-6-x86_64,fedora-7-x86_64,fedora-8-x86_64,fedora-9-x86_64,fedora-10-x86_64,rhel-4-x86_64,rhel-5-x86_64,suse-10.2-x86_64,suse-10.3-x86_64,suse-11.0-x86_64,suse-11.1-x86_64,sles-10-x86_64,gentoo-nover-x86_64,debian-4.0-x86_64,debian-5.0-x86_64,ubuntu-7.04-x86_64,ubuntu-7.10-x86_64,ubuntu-8.04-x86_64,ubuntu-8.10-x86_64,ubuntu-9.04-x86_64
Packages: netperf
[...]
$ find . -name netperf'*'
./fedora/4/netperf-2.4.5-1.fc4.i386.rpm
./fedora/4/netperf-2.4.5-1.fc4.src.rpm
[...]
./mandrake/10.2/netperf-2.4.5-1.mdk102.i586.rpm
./mandrake/10.2/netperf-2.4.5-1.mdk102.src.rpm
[...]
./redhat/7.3/netperf-2.4.5-1.rh73.i386.rpm
./redhat/7.3/netperf-2.4.5-1.rh73.src.rpm
[...]
./rhel/5/netperf-2.4.5-1.rhel5.i386.rpm
./rhel/5/netperf-2.4.5-1.rhel5.src.rpm
./rhel/5/netperf-2.4.5-1.rhel5.x86_64.rpm
[...]
./src/netperf-latest.tar.gz
./src/netperf-2.4.5.tar.gz
[...]
./suse/10.3/netperf-2.4.5-1.suse10.3.i586.rpm
./suse/10.3/netperf-2.4.5-1.suse10.3.src.rpm
./suse/10.3/netperf-2.4.5-1.suse10.3.x86_64.rpm

So we now have rpms built for all the distributions supported.

Last modified 5 years ago Last modified on May 13, 2012, 12:00:42 PM