Changeset 1549


Ignore:
Timestamp:
May 21, 2012, 2:17:30 AM (7 years ago)
Author:
bruno
Message:
  • pb: Improve build2pkg documentation so It can be referenced in build2ve. Document need to specify verpmtype for a rpm-style VE. Add substep of further fixes to filters and build files in build2ve. Provide example of needing an additional repo, which is more likely on centos. Fix bug with adapting owner since centos5.8 sets up so that we can't sudo until the sudoers file is fixed, but we don't do so until later. However, if we call su, then we need to keep the additional sudo. Apply the ftp/http proxy bits from the config file while setting up a VE. Don't try to mount /proc if it is already mounted. Fiddle with the substitution on requiretty; it wasn't working so moved to \s+ instead of [ \t]+ also left a comment to show users that the line was removed. Remove comment about applying ftp/http proxy to bootstrapping since we now do. Add note that we probably want a variable for where to get the source snapshot for source "installs" of project-builder.
  • Filter.pm: Add support for multi-line filter variables since a description in an rpm or deb file is usually multiple lines long (this is now in addition to the support of the $/ variable). Add support for transforming those things since debian files use leading spaces, and rpm files don't. Transforms let you start with the same variable and fix it for the different OSs. Leave note about unknown "correct" documentation location.
Location:
devel/pb
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • devel/pb/bin/pb

    r1548 r1549  
    492492=head2 build2pkg
    493493
    494 The build2pkg command takes the tar files created in the cms2build step and attempts to build the binary packages for your current operating system. In order for this step to work, you may need to edit the files in one of the $pbdefdir/$project/pbconf/$version/$project/{deb,rpm,pkg} directories. Those files will be used to try to build your package. Note that if you change those files, you need to re-run the cms2build step. Then you can run a command like:
     494The build2pkg command takes the tar files created in the cms2build step and attempts to build binary packages for your current operating system. There are two steps:
     495
     496=over 4
     497
     498=item Update your filters and build files.
     499
     500You probably need to edit the files describing the build steps in one of the $pbdefdir/$project/pbconf/$version/$project/{deb,rpm,pkg} directories and the filters in $pbdefdir/$project/pbconf/$version/pbfilter. Note that you can define additional filters and transforms in the *.pbf files. The build files will be filtered by the filters defined in the *.pbf files to generate the inputs to the build step. Therefore, if you change those files, you need to re-run the cms2build step.
     501
     502=item Build the package.
     503
     504Then you can run a command like:
    495505
    496506 % pb -p $project -r $version build2pkg
    497507
    498508To create the files in $project/build that comprise your binary package(s).
     509
     510=back
    499511
    500512=head2 newve
     
    513525 vetype default = chroot
    514526
     527If you are building for rpm style OS's, update the verpmtype option, and install the appropriate tool.
     528
     529 verpmtype default = rpmbootstrap
     530
    515531You may also choose to specify a mirror for the OS packages, and optionally http/ftp proxies.  You can specify the proxies either through environment variables ($http_proxy/$ftp_proxy) or in the configuration file. The configuration file will be used if no corresponding environment variable has been set. For example, for debian and with a local squid proxy:
    516532
     
    555571The build2ve command is similar to the build2pkg command in that it will take the sources created by cms2build and turn them into binary packages. The command has two differences. First, it creates the packages in a virtual environment, i.e. the one made by an earlier setupve setup. Second it copies the resulting packages to a repository and builds the repository meta-data needed.
    556572
    557 Two sub-steps are needed:
     573Three sub-steps are needed:
    558574
    559575=over 4
     
    565581 sshhost $project = localhost
    566582 sshdir $project = $home/cache-project-builder/repos
     583
     584You may also need to specify additional repository files to use or rpms to install. Note the URL for repositories is not the URL of the repository, but the URL of a file that can be put in the yum.repos.d or apt.sources.d directory.
     585
     586 addrepo centos-5-i386 = http://localhost/pb/centos-extras.repo,http://mirror.centos.org/centos/5/extras/i386/RPMS/chrpath-0.13-3.el5.centos.i386.rpm
     587
     588=item Update your filters and build files
     589
     590You may need to update your filter files (*.pbf) as in the build2pkg step if you are building for a new OS or architecture.
    567591
    568592=item Build the packages and copy them to the repository
     
    21652189    # ssh communication if not VE or CPAN
    21662190    # should use a hash instead...
    2167     my ($shcmd,$cpcmd,$cptarget,$cp2target);
     2191    my ($shcmd,$shcmdroot,$cpcmd,$cptarget,$cp2target);
    21682192    if ($cmt =~ /^VE/) {
    21692193        my $tp = pb_path_expand($vepath->{$ENV{'PBPROJ'}});
     
    21732197        my $arch = pb_get_arch();
    21742198        if ($vetype eq "chroot") {
    2175             $shcmd = "sudo /usr/sbin/chroot $tpdir /bin/su - $mac -c ";
     2199            $shcmdroot = "sudo /usr/sbin/chroot $tpdir ";
     2200            $shcmd = "$shcmdroot /bin/su - $mac -c ";
    21762201        } elsif ($vetype eq "schroot") {
    21772202            $shcmd = "schroot $tp -u $mac -- ";
    21782203        }
    2179         $sshcmd = "setarch i386 $sshcmd" if (($pbos->{'arch'} =~ /i?86/) && ($arch eq 'x86_64'));
     2204        $shcmd = "setarch i386 $shcmd" if (($pbos->{'arch'} =~ /i?86/) && ($arch eq 'x86_64'));
    21802205        $cpcmd = "sudo /bin/cp -r ";
    21812206        # We need to get the home dir of the target account to deliver in the right place
     
    22332258    # For VE we need to change the owner manually
    22342259    if ($cmt =~ /^VE/) {
    2235         pb_system("$shcmd \"sudo chown -R $mac $tdir\"","Adapt owner in $tdir to $mac");
     2260        if (defined $shcmdroot) {
     2261            # This should help overcome a CentOS 5.8 bug as well as having a simper sequence
     2262            pb_system("$shcmdroot \"chown -R $mac $tdir\"","Adapt owner in $tdir to $mac");
     2263        } else {
     2264            pb_system("$shcmd \"sudo chown -R $mac $tdir\"","Adapt owner in $tdir to $mac");
     2265        }
    22362266    }
    22372267
     
    24212451    my $pbos = pb_distro_get_context($v);
    24222452   
     2453    my $ftp_proxy = pb_distro_get_param($pbos,pb_conf_get_if("ftp_proxy"));
     2454    my $http_proxy = pb_distro_get_param($pbos,pb_conf_get_if("http_proxy"));
     2455
     2456    $ENV{ftp_proxy} ||= $ftp_proxy if ((defined $ftp_proxy) && ($ftp_proxy ne ""));
     2457    $ENV{http_proxy} ||= $http_proxy if ((defined $http_proxy) && ($http_proxy ne ""));
     2458
    24232459    # Launch the VMs/VEs
    24242460    if ($vtype eq "vm") {
     
    29953031
    29963032# For VE we first need to mount some FS
    2997 pb_system("mount -t proc /proc /proc");
     3033pb_system("mount -t proc /proc /proc") unless (-d "/proc/$$");
    29983034
    29993035EOF
     
    31533189EOF
    31543190    print SCRIPT << 'EOF';
    3155     s/Defaults[ \t]+requiretty//;
     3191    s/Defaults\s+requiretty/# disable Defaults: requiretty/;
    31563192    print PBOUT $_;
    31573193}
     
    32053241    }
    32063242    # We may need a proxy configuration. Get it from the local env
    3207 
    3208     # TODO: make this apply to the bootstrapping also.
    3209     my ($ftp_proxy_map, $http_proxy_map) = pb_conf_get_if('ftp_proxy', 'http_proxy');
    3210     my $ftp_proxy = pb_distro_get_param($pbos, $ftp_proxy_map);
    3211     my $http_proxy = pb_distro_get_param($pbos, $http_proxy_map);
    3212 
    3213     $ENV{'ftp_proxy'} = $ftp_proxy if ((defined $ftp_proxy) && ($ftp_proxy ne '') && (not defined $ENV{'ftp_proxy'}));
    3214     $ENV{'http_proxy'} = $http_proxy if ((defined $http_proxy) && ($http_proxy ne '') && (not defined $ENV{'http_proxy'}));
     3243    my $ftp_proxy = pb_distro_get_param($pbos,pb_conf_get_if("ftp_proxy"));
     3244    my $http_proxy = pb_distro_get_param($pbos,pb_conf_get_if("http_proxy"));
     3245
     3246    $ENV{'ftp_proxy'} ||= $ftp_proxy if ((defined $ftp_proxy) && ($ftp_proxy ne ""));
     3247    $ENV{'http_proxy'} ||= $http_proxy if ((defined $http_proxy) && ($http_proxy ne ""));
    32153248
    32163249    if (defined $ENV{'http_proxy'}) {
     
    32673300            $cmdget = "mv /tmp/ProjectBuilder-$pbver1$pbextdir.tar.gz ProjectBuilder-latest.tar.gz ; mv /tmp/project-builder-$pbver2$pbextdir.tar.gz project-builder-latest.tar.gz";
    32683301        } else {
     3302            # TODO: shouldn't we have a variable to specify the source file?
     3303            # And shouldn't we be getting the one that matches the current pb version?
     3304            # For the moment, just take the latest stable one
    32693305            $cmdget = "wget --passive-ftp ftp://ftp.project-builder.org/src/ProjectBuilder-latest.tar.gz; wget --passive-ftp ftp://ftp.project-builder.org/src/project-builder-latest.tar.gz";
    32703306        }
  • devel/pb/lib/ProjectBuilder/Filter.pm

    r1434 r1549  
    119119        while(<CONF>)  {
    120120            if (/^\s*([A-z0-9-_]+)\s+([[A-z0-9-_]+)\s*=\s*(.+)$/) {
    121                 pb_log(3,"DEBUG creating entry $1, key $2, value $3\n");
    122                 $h{$1}{$2}=$3;
     121                my ($what, $var, $value) = ($1, $2, $3);
     122                pb_log(3,"DEBUG creating entry $what, key $var, value $value\n");
     123                # Add support for multi-lines
     124                while ($value =~ s/\\\s*$//o) {
     125                    $_ = <CONF>;
     126                    die "Still processing continuations for $what $var at EOF" if (not defined $_);
     127                    s/[\r\n]//go;
     128                    $value .= "\n$_";
     129                }
     130                $h{$what}{$var}=$value;
     131            } elsif ((/^\s*#/o) || (/^\s*$/o)) {
     132                # ignore
     133            } else {
     134                chomp();
     135                warn "unexpected line '$_' in $f";
    123136            }
    124137        }
     
    126139    }
    127140    $ptr = $h{"filter"};
     141
     142    # TODO: find a place to document it. Why not in this file as pod and also documenting filters ?
     143    # Handle transform
     144    if (defined $h{transform}) {
     145        while (my ($out_key,$spec) = each %{$h{transform}}) {
     146            die "Unknown transform for $out_key '$spec' expected <out-key> <transform>" unless $spec =~ /^([\w\-]+)\s+(.+)$/;
     147            my ($in_key, $expr) = ($1, $2);
     148            local $_ = $ptr->{$in_key} || '';
     149            eval $expr;
     150            die "Error evaluating tranform for $out_key ($expr): $@" if $@;
     151            $ptr->{$out_key} = $_;
     152            pb_log(2, "Transform $in_key to $out_key\n$ptr->{$in_key}\n$ptr->{$out_key}\n");
     153        }
     154    }
    128155}
    129156pb_log(2,"DEBUG f:".Dumper($ptr)."\n") if (defined $ptr);
Note: See TracChangeset for help on using the changeset viewer.