Changes between Version 19 and Version 20 of WikiStart


Ignore:
Timestamp:
Nov 16, 2007, 1:00:38 AM (13 years ago)
Author:
Bruno Cornec
Comment:

perfmon up to cms2pkg without good result

Legend:

Unmodified
Added
Removed
Modified
  • WikiStart

    v19 v20  
    3333}}}
    3434
     35Should be obvious for the content. Note that the format is in general
     36{{{
     37pb-keyword project = value
     38}}}
     39which means that here I named my project for netperf for Project-Builder (I can choose what I want but need to be consistent everywhere).
     40
    3541Then get the bits of the project you want to package. It could be using a SVN or CVS access if there is such a thing, or by downloading (at worse) the tar file of the project. Then extract that under a newly created directory that you referenced in your project configuration file:
    3642
     
    3844$ cd /tmp
    3945$ wget ftp://ftp.netperf.org/netperf/netperf-2.4.4.tar.bz2
    40 $ mkdir $HOME/netperf/src
     46$ mkdir -p $HOME/netperf/src
    4147$ cd $HOME/netperf/src
    4248$ tar xvfj /tmp/netperf-2.4.4.tar.bz2
    43 }}}
    44 
    45 = Example of use on LinuxCOE (more complex example) =
    46 
    47 When the project was mature enough to package correctly mondorescue and itself, I then decided it was time to try it on another unrelated project to see how difficult it could be and document that experience so that others could also begin to use pb for their own project.
    48 
    49 You first have to edit your .pbrc configuration file in your home directory, which will setup things globally.
    50 
    51 Here are my .pbrc LinuxCOE relative entries:
    52 {{{
    53 $ grep -E '^#|linuxcoe|default' ~/.pbrc
    54 #
    55 # Root of the temp file used to store info on the project
    56 # Is also used to determine the root directory of the project
    57 # No default option allowed here as they need to be all different
    58 #
    59 pbrc linuxcoe = /users/bruno/LinuxCOE/cvs/pbrc
    60 #
    61 # Which CMS system is used
    62 #
    63 cms linuxcoe = cvs
    64 cms default = svn
     49$ rm -f /tmp/netperf-2.4.4.tar.bz2
     50}}}
     51
     52The netperf project is a nice project using a simple environment to build itself. It uses the GNU autoconf/autmake/litool mecanism, which makes it easy to package with pb.
     53
     54You will now need to get the latest and greatest project-builder software from our [ftp://ftp.project-builder.org ftp] repository. Of course we provide lots of packages for your distribution ;-) Or a tar.gz file if you prefer. Once available, you can use the newproj option of pb to create the infrastructure that is mandatory for pb. It is mandatory that there is a correspondance between the directory mentioned in $HOME/.pbrc and the -r option:
     55
     56{{{
     57$ pb -p netperf -r `pwd` newproj
     58Creating /users/bruno/netperf/src/pbconf directory
     59
     60Do not to forget to commit the pbconf directory in your CMS if needed
     61After having renamed the pkg1 directory to your package's name
     62
     63Project: netperf
     64Action: newproj
     65}}}
     66
     67Explore then the pbconf directory to familiarize yourself with some of the config file. the next one we want to edit, is the main configuration file which will be used as the central piece of information for pb.
     68
     69Edit it and adapt it. After edition, here is the resulting file:
     70{{{
     71$ ls -R pbconf
     72pbconf:
     73netperf.pb  pbfilter  pkg1
     74
     75pbconf/pbfilter:
     76all.pbf  deb.pbf  md.pbf  novell.pbf  rpm.pbf
     77
     78pbconf/pkg1:
     79deb  pbfilter  rpm
     80
     81pbconf/pkg1/deb:
     82changelog  compat  control  copyright  pkg1.dirs  pkg1.docs  rules
     83
     84pbconf/pkg1/pbfilter:
     85
     86pbconf/pkg1/rpm:
     87pkg1.spec
     88$
     89$ vi pbconf/netperf.pb
     90[Modifications here]
     91$ cat pbconf/netperf.pb
     92#
     93# Project Builder configuration file
     94# For project netperf
     95#
     96# $Id$
     97#
     98
     99#
     100# Which CMS system is used (Subversion, CVS or tar file content extracted)
     101#
     102#cms netperf = svn
     103#cms netperf = cvs
     104cms netperf = flat
     105
    65106#
    66107# Packager label
    67108#
    68 packager linuxcoe = "Bruno Cornec <bruno@project-builder.org>"
    69 #
    70 # For CVS managed projects
    71 # (Optional)
    72 #
    73 cvsroot linuxcoe = ":pserver:anonymous@linuxcoe.cvs.sourceforge.net:/cvsroot/linuxcoe"
     109packager netperf = "Bruno Cornec <bruno@project-builder.org>"
     110#
     111
     112# For delivery to a machine by SSH (potentially the FTP server)
     113# Needs hostname, account and directory
     114#
     115sshhost netperf = www.project-builder.org
     116sshlogin netperf = bruno
     117sshdir netperf = /mondo/ftp
     118sshport netperf = 22
     119
    74120#
    75121# For Virtual machines management
     
    77123# followed by '_' and by release number
    78124# a .vmtype extension will be added to the resulting string
    79 # a rhel_3 here means that the VM will be named rhel_3.qemu
    80 #
    81 vmlist linuxcoe = mandriva_2007.1
     125# a QEMU rhel_3 here means that the VM will be named rhel_3.qemu
     126#
     127vmlist netperf = mandrake_10.1,mandrake_10.2,mandriva_2006.0,mandriva_2007.0,mandriva_2007.1,mandriva_2008.0,redhat_7.3,redhat_9,fedora_4,fedora_5,fedora_6,fedora_7,rhel_3,rhel_4,rhel_5,suse_10.0,suse_10.1,suse_10.2,suse_10.3,sles_9,sles_10,gentoo_nover,debian_3.1,debian_4.0,ubuntu_6.06,ubuntu_7.04,ubuntu_7.10
     128
    82129#
    83130# Valid values for vmtype are
    84131# qemu, (vmware, xen, ... TBD)
    85 vmtype default = qemu
    86 }}}
    87 
    88 Should be obvious for the content. Note that the format is in general
    89 {{{
    90 pb-keyword project = value
    91 }}}
    92 which means that here I named my project for pb linuxcoe (I can choose what I want be need to be consistent everywhere).
     132vmtype netperf = qemu
     133
     134# Hash for VM stuff on vmtype
     135vmntp default = pool.ntp.org
     136
     137# We suppose we can commmunicate with the VM through SSH
     138vmhost netperf = localhost
     139vmlogin netperf = pb
     140vmport netperf = 2225
     141
     142# Timeout to wait when VM is launched/stopped
     143vmtmout default = 120
     144
     145# per VMs needed paramaters
     146vmopt netperf = -m 384 -daemonize
     147vmpath netperf = /home/qemu
     148vmsize netperf = 5G
     149
     150#
     151# Global version/tag for the project
     152#
     153projver netperf = 2.4.4
     154projtag netperf = 1
     155
     156# Adapt to your needs:
     157# Optional if you need to overwrite the global values above
     158#
     159#pkgver pkg1 = stable
     160#pkgtag pkg1 = 3
     161#pkgver nil
     162#pkgtag nil
     163
     164# Hash of default package/package directory
     165defpkgdir netperf = netperf-2.4.4
     166
     167# Hash of additional package/package directory
     168#extpkgdir pkg1-doc = pkg1-docdir
     169
     170# Hash of valid version names
     171version devel
     172#version stable
     173
     174# List of files per pkg on which to apply filters
     175# Files are mentioned relatively to pbroot/defpkgdir
     176filteredfiles netperf = configure.ac
     177#filteredfiles pkg1-doc = configure.in
     178}}}
     179I skip the parameters begining with vm and ssh for the moment. I inform pb that I will produce packages with a version-tag of 2.4.4-1, that no local package should have a different version than the project version (pkgver/pkgtag), that my default package will be netperf (pb will work on it by default all the time) and that no additional packages will be produced. In addition to standard version with numbers, pb will accept the version devel. And finally, I give to pb the name of all files (per package) that it will have to filter to produce the packages - more on that later on.
     180Once this is done, you may already try to see if pb can make something useful for you. Check the result of:
     181
     182{{{
     183$ pb -p netperf -r `pwd` cms2build
     184Project: netperf
     185Action: cms2build
     186Packages: netperf
     187
     188Management of netperf 2.4.4-1 (rev flat)
     189Exporting /users/bruno/netperf/src/netperf-2.4.4 from DIR to /users/bruno/netperf/delivery/netperf-2.4.4... OK
     190Build files generated for
     191No Build files found for mandriva-2006.0,sles-9,mandrake-10.2,mandriva-2007.0,rhel-3,mandriva-2007.1,redhat-9,ubuntu-6.06,suse-10.3,rhel-5,ubuntu-7.10,gentoo-nover,suse-10.2,mandrake-10.1,mandriva-2008.0,sles-10,fedora-6,suse-10.1,fedora-5,fedora-4,ubuntu-7.04,fedora-7,suse-10.0,debian-4.0,rhel-4,debian-3.1,redhat-7.3
     192filter: no such variable
     193Can't use an undefined value as a HASH reference at /usr/lib/perl5/vendor_perl/5.8.8/ProjectBuilder/Base.pm line 1077.
     194}}}
     195Ok we tried to make a build file from ou Configuration Management System (CMS) which didn't work completely as our structure is incomplete. However, pb already recognized the project name, action, package to build, type of CMS (flat), used our -r option to create the intermediate delivery directory where all the final files|packages will be created.
     196
     197So now, in order to be more useful, we need to create the structure for our package. Luckily, pb did most of the work for us, by creating a template pkg1 directory and files under pbconf. Rename them to netperf (name of our package) and explore and adapt its content:
     198
     199{{{
     200$ find . -name pkg1'*'
     201./pbconf/pkg1
     202./pbconf/pkg1/rpm/pkg1.spec
     203./pbconf/pkg1/deb/pkg1.dirs
     204./pbconf/pkg1/deb/pkg1.docs
     205$ mv ./pbconf/pkg1 ./pbconf/netperf
     206$ mv ./pbconf/netperf/rpm/pkg1.spec ./pbconf/netperf/rpm/netperf.spec
     207$ mv ./pbconf/netperf/deb/pkg1.dirs ./pbconf/netperf/deb/netperf.dirs
     208$ mv ./pbconf/netperf/deb/pkg1.docs ./pbconf/netperf/deb/netperf.docs
     209}}}
     210
     211Now this is done, it's time to look at those and adapt them. At the begining, we try to make RPM packages.
     212{{{
     213$ vi pbconf/netperf/rpm/netperf.spec
     214[Modifications here]
     215$ cat pbconf/netperf/rpm/netperf.spec
     216#
     217# $Id$
     218#
     219
     220Summary:        The netperf tool is a benchmarking network tool
     221Summary(fr):    netperf est un outil de benchmark réseau
     222
     223Name:           PBPKG
     224Version:        PBVER
     225Release:        PBTAGPBSUF
     226License:        GPL
     227Group:          PBGRP
     228Url:            PBURL
     229Source:         PBSRC
     230BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(id -u -n)
     231Requires:       PBDEP
     232
     233%description
     234PBDESC
     235
     236%description -l fr
     237netperf est un outil de benchmark réseau
     238
     239%prep
     240%setup -q
     241
     242%build
     243%configure
     244make %{?_smp_mflags} VERSION=%{version}
     245
     246%install
     247%{__rm} -rf $RPM_BUILD_ROOT
     248make DESTDIR=$RPM_BUILD_ROOT install
     249
     250%clean
     251%{__rm} -rf $RPM_BUILD_ROOT
     252
     253%files
     254%defattr(-,root,root)
     255%doc ChangeLog
     256%doc INSTALL COPYING README AUTHORS NEWS
     257
     258%changelog
     259PBLOG
     260}}}
     261
     262That spec file looks normal except that it uses a lot of PBSOMETHING keywords. Those are Project-Builder macros that will be instantiate depending on the distribution during the cms2build step.
     263
     264So now that we have the base for our spec file, we need to look at those filters:
     265
     266{{{
     267$ vi pbconf/pbfilter/all.pbf
     268[Modifications here]
     269$ cat pbconf/pbfilter/all.pbf
     270#
     271# $Id$
     272#
     273# Filter for all files
     274#
     275# PBSRC is replaced by the source package format
     276filter PBSRC = ftp://ftp.netperf.org/netperf/%{name}-%{version}.tar.gz
     277
     278# PBVER is replaced by the version ($pbver in code)
     279filter PBVER = $pbver
     280
     281# PBDATE is replaced by the date ($pbdate in code)
     282filter PBDATE = $pbdate
     283
     284# PBLOG is replaced by the changelog if value is yes
     285filter PBLOG = yes
     286
     287# PBTAG is replaced by the tag ($pbtag in code)
     288filter PBTAG = $pbtag
     289
     290# PBREV is replaced by the revision ($pbrev in code)
     291filter PBREV = $pbrev
     292
     293# PBPKG is replaced by the package name ($pbpkg in code)
     294filter PBPKG = $pbpkg
     295
     296# PBPACKAGER is replaced by the packager name ($pbpackager in code)
     297filter PBPACKAGER = $pbpackager
     298
     299# PBDESC contains the description of the package
     300filter PBDESC = "netperf is a network benchmarking tool"
     301
     302# PBURL contains the URL of the Web site of the project
     303filter PBURL = http://www.netperf.org
     304}}}
     305We can now try again to use pb to create our build file:
     306{{{
     307$ pb -p netperf -r `pwd` cms2build
     308Project: netperf
     309Action: cms2build
     310Packages: netperf
     311
     312Management of netperf 2.4.4-1 (rev flat)
     313Exporting /users/bruno/netperf/src/netperf-2.4.4 from DIR to /users/bruno/netperf/delivery/netperf-2.4.4... OK
     314Build files generated for mandriva-2006.0,sles-9,mandrake-10.2,mandriva-2007.0,rhel-3,mandriva-2007.1,redhat-9,ubuntu-6.06,suse-10.3,rhel-5,ubuntu-7.10,suse-10.2,mandrake-10.1,mandriva-2008.0,sles-10,fedora-6,suse-10.1,fedora-5,fedora-4,ubuntu-7.04,fedora-7,suse-10.0,debian-4.0,rhel-4,debian-3.1,redhat-7.3
     315No Build files found for gentoo-nover
     316Files configure.ac have been filtered
     317Creating netperf tar files compressed... OK
     318Under /users/bruno/netperf/delivery/netperf-2.4.4.tar.gz
     319}}}
     320
     321Much better. You have now your first useful result from pb :-) Congrats !
     322
     323= Example of use on LinuxCOE (more complex example) =
     324
     325When the project was mature enough to package correctly mondorescue and itself, I then decided it was time to try it on another unrelated project to see how difficult it could be and document that experience so that others could also begin to use pb for their own project.
     326
     327You first have to edit your .pbrc configuration file in your home directory, which will setup things globally.
     328
     329Here are my .pbrc LinuxCOE relative entries:
     330{{{
     331$ grep -E '^#|linuxcoe|default' ~/.pbrc
     332#
     333# Root of the temp file used to store info on the project
     334# Is also used to determine the root directory of the project
     335# No default option allowed here as they need to be all different
     336#
     337pbrc linuxcoe = /users/bruno/LinuxCOE/cvs/pbrc
     338}}}
    93339
    94340Then you have to go under the directory you planned to store the project you want to package (/users/bruno/LinuxCOE/cvs) and you have to create there a directory pbconf which will contain everything related to project-builder around your own project. The pbrc file you indicated in the configuation file will be managed automatically by pb and you don't have to care of it. Just to reference where it will be, which will in turn give to project-builder the information of the project's directory.
     
    142388}}}
    143389
    144 I inform pb that I will produce packages with a verion-tag of devel-1, that no local package should have a different version than the project version (pkgver/pkgtag), that my default package will be systemdesigner (pb will work on it by default all the time) and that additional packages it may deal with are systemdesigner-docs and systemdesigner-fedora. In addition to standard version with numbers, pb will accpet the version devel and stable. And finally, I give to the name of all files (per package) that it will have to filter to produce the packages - more on that later on.
     390I inform pb that I will produce packages with a version-tag of devel-1, that no local package should have a different version than the project version (pkgver/pkgtag), that my default package will be systemdesigner (pb will work on it by default all the time) and that additional packages it may deal with are systemdesigner-docs and systemdesigner-fedora. In addition to standard version with numbers, pb will accpet the version devel and stable. And finally, I give to the name of all files (per package) that it will have to filter to produce the packages - more on that later on.
    145391
    146392With that minimal configuration, you may begin to test project-builder. Try to produce a compressed tar file of your project.