Monday, September 24, 2012

SVr4 relocatable package

A relocatable package can be more easily developed then a non-relocatable package.
This is probably because it tends to be more self-contained.
The very first steps are common task.
  
# zfs create -o mountpoint=/pkg data/pkg
# zfs create data/pkg/src
# zfs create data/pkg/out
# mkdir /pkg/src/PHXapp1
# cd /pkg/src/PHXapp1
 
The main idea of a relocatable package is that of a base directory.
It's the root of a directory hierarchy comprising all of the package contents.
It's highly recommended to choose somewhere under /opt and not /usr/local.
Also organize the directory hierarchy, contents, ownership and permissions.
(metafiles are highlighted below just to make them better distinguishable)
 
# cp -pr /project/PHXapp1/* .
# ll
total ...
drwxr-xr-x   2 root  bin         4 Sep 24 11:38 bin
drwxr-xr-x   2 root  bin         3 Sep 24 11:38 include
drwxr-xr-x   2 root  bin         3 Sep 24 11:39 lib
drwxr-xr-x   2 root  bin         3 Sep 24 11:38 man

-rw-r--r--   1 root  root      186 Sep 25 14:40 copyright
-rw-r--r--   1 root  root      236 Sep 28 07:52 pkginfo
-rw-r--r--   1 root  root      381 Sep 28 10:19 request

-rw-r--r--   1 root  root       69 Sep 28 08:01 postremove
-rw-r--r--   1 root  root      333 Sep 28 08:26 prototype

  
Most of what's left to do is quite similar to the non-relocatable package's creation.
The first noticeable exception is the pkginfo's BASEDIR variable.
  
Note that by omitting BASEDIR will make pkgkadd emit an empty prompt for it.
Don't omit BASEDIR to make it a default to a request script base directory prompt.

Interestingly the BASEDIR is not automatically removed by pkgrm.
As the package is relocatable, the base directory may vary among installations.
Thus it isn't a good idea to hard code the base directory in the prototype either.
There comes in the postremove script together with the CLIENT_BASEDIR variable.
  
# cat pkginfo
PKG="PHXapp1"
NAME="Relocatable sample package"
ARCH="i386,sparc"
VERSION="1.0"
CATEGORY="system"
DESC="Relocatable package demonstration"
VENDOR="AZ - Learnings on Solaris"
PSTAMP="Sep 24, 2012"
CLASSES="none"  
BASEDIR=/opt/PHXapp1
  
# cat prototype
! search bin include lib man
i copyright
i pkginfo
i request
i postremove
d none bin 0755 root sys
f none bin/script 0750 root root
f none bin/app1 0550 root sys
d none lib 0755 root sys
f none lib/libapp1 0444 root root
d none include 0755 root sys
f none include/app1.h 0444 root root
d none man 0755 root sys
f none man/man1 0644 root root

# cat request
#!/bin/sh
   
while [ 1 ]
do
    printf "\nPath to base directory (q to quit) [$BASEDIR] "
    read DIRECTORY
   
    case "$DIRECTORY" in
 
        "") break;
            ;;
   
        /*) BASEDIR="$DIRECTORY"
            break;
            ;;
   
         q) exit 1
            ;;
  
         *) printf "\tERROR: Path must begin with a slash (/).\n"
            ;;
    esac
done
 
cat >$1 <EOF
BASEDIR=$BASEDIR
EOF
 
exit 0

   
# cat postremove
#!/bin/sh
echo Removing package base directory.
# Let the system complain if directory not empty (as expected).
rmdir $CLIENT_BASEDIR
exit 0
  
# pkgmk -o -d /pkg/out
## Building pkgmap from package prototype file.
## Processing pkginfo file.
## Attempting to volumize 9 entries in pkgmap.
part  1 -- 22 blocks, 21 entries
## Packaging one part.
/pkg/out/PHXapp1/pkgmap
/pkg/out/PHXapp1/pkginfo
/pkg/out/PHXapp1/reloc/bin/app1
/pkg/out/PHXapp1/reloc/bin/script
/pkg/out/PHXapp1/install/copyright
/pkg/out/PHXapp1/reloc/include/app1.h
/pkg/out/PHXapp1/reloc/lib/libapp1
/pkg/out/PHXapp1/reloc/man/man1
/pkg/out/PHXapp1/install/postremove
/pkg/out/PHXapp1/install/request
## Validating control scripts.
## Packaging complete.
 
  
# pkgtrans /pkg/out /pkg/out/PHXapp1.pkg PHXapp1
 
# ll /pkg/out 
total ...
drwxr-xr-x   3 root root       5 Sep 24 11:39 PHXapp1
-rw-r--r--   1 root root    4.0K Sep 24 12:25 PHXapp1.pkg

...

A few verifications could be:
   
# pkgchk -d /pkg/out/PHXapp1.pkg all
Checking uninstalled stream format package <PHXapp1> from </pkg/out/PHXapp1.pkg>
## Checking control scripts.
## Checking package objects.
## Checking is complete.
 
# pkginfo -l -d /pkg/out/PHXapp1.pkg
   PKGINST:  PHXapp1
      NAME:  Relocatable sample package
  CATEGORY:  system
      ARCH:  i386,sparc
   VERSION:  1.0
   BASEDIR:  /opt/PHXapp1
    VENDOR:  AZ - Learnings on Solaris
      DESC:  Relocatable package demonstration
    PSTAMP:  Sep 24, 2012
    STATUS:  spooled
     FILES:       14 spooled pathnames
                   4 directories
                   2 executables
                   5 package information files
                   5 blocks used (approx)

  

Thursday, September 20, 2012

Custom JumpStart first boot

Assume that the setup is according to Custom JumpStart framework setup.
Once the base system is installed, additional customization can take place.
In order to achieve this, an script must be run once on the first boot.
    
The options are:
 
  • A traditional self-destructive initialization script.
    This may be simpler but it's deprecated, so better avoid it.
        
  • A transient self-destructive SMF service.
    The overall benefits of SMF tend to compensate the hurdles.  

For manageability, wrap the SMF service on a non-relocatable SVr4 package.
Make sure to avoid all interactions during package installation and removal.
This way it can be included in the Custom JumpStart general conveniences.
   
Assume such a package named AZ-1st-boot containing the following:
 
  • A transient service manifest, AZ-1st-boot.xml, describing the transient service.
    To be automatically imported into services database on the next reboot,
    place this file in /var/svc/manifest/site.
         
  • The first boot script, AZ-1st-boot.sh, is invoked by the transient service. 
    The /opt directory is the only correct place to put additional software.
    The /usr/local is not recommended for anything in Solaris.

IMPORTANT
Incidentally for a first boot script the class manifest must not be used.
If in the correct place, the SMF service manifest will be imported on the first boot.

As an example, each file could be as follows:
  
# cd /pkg/src/AZ-1st-boot
  
# cat pkginfo
PKG="AZ-1st-boot"
NAME="AZ first boot"
ARCH="i386,sparc"
VERSION="1.0"
CATEGORY="system"
DESC="A 1st-boot one-time-only SMF service for finishing installation"
VENDOR="AZ - Learnings on Solaris"
PSTAMP="Oct 11, 2012"
CLASSES="none"
  
# cat prototype 
i pkginfo
f none /var/svc/manifest/site/AZ-1st-boot.xml 0444 root sys

d none /opt/site 0755 root sys
f none /opt/site/AZ-1st-boot.sh 0555 root sys
 
# cat AZ-1st-boot.xml
<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='AZ:1st-boot-svc'>
<service name='site/1st-boot-svc' type='service' version='1'>

<create_default_instance enabled='true'/>
<single_instance/>

<dependency 
    name='multi-user' 
    grouping='require_all' 
    restart_on='none' type='service'>
        <service_fmri value='svc:/milestone/multi-user:default'/>
</dependency>

<exec_method
    type='method'
    name='start'
    exec='/opt/site/AZ-1st-boot.sh'
    timeout_seconds='300'
>
        <method_context>
            <method_credential user='root'/>
        </method_context>
</exec_method>

<exec_method
    type='method'
    name='stop'
    exec=':true'
    timeout_seconds='30'/
>

<property_group name='startd' type='framework'>
    <propval name='duration' type='astring' value='transient'/>
</property_group>

</service>
</service_bundle>
 

# cat AZ-1st-boot.sh
#!/bin/sh
  
. /lib/svc/share/smf_include.sh
  
BIN=/usr/bin
SBIN=/usr/sbin
  
#.................................................................
# configure and enable SAR
  
$BIN/ed -s /var/spool/cron/crontabs/sys <<EOF
a
0,5,10,15,20,25,30,35,40,45,50,55 * * * 0-6 /usr/lib/sa/sa1
59 23 * * 0-6 /usr/lib/sa/sa2 -A
.
w
q
EOF

$SBIN/svcadm enable svc:/system/sar:default
  
#.................................................................
# configure FSS scheduler
 
$SBIN/dispadmin -d FSS
  
#.................................................................
# self-destroy
 

$SBIN/svcadm disable svc:/site/1st-boot-svc:default
$SBIN/svccfg delete -f svc:/site/1st-boot-svc:default
    
$BIN/ed -s <<EOF
a
mail=
setuid=nocheck
action=nocheck
.
w /tmp/AZ.admin
q
EOF
  
$SBIN/pkgrm -n -a /tmp/AZ.admin AZ-1st-boot
  
#.................................................................
# end
 
$SBIN/shutdown -i6 -g5 -y
   
exit $SMF_EXIT_OK

# pkgmk -o -d /package
## Building pkgmap from package prototype file.
## Processing pkginfo file.
WARNING: missing directory entry for
</opt>
WARNING: missing directory entry for </var>
WARNING: missing directory entry for </var/svc>
WARNING: missing directory entry for </var/svc/manifest>
WARNING: missing directory entry for </var/svc/manifest/site>
## Attempting to volumize 3 entries in pkgmap.
part  1 -- 18 blocks, 14 entries
## Packaging one part.
/package/AZ-1st-boot/pkgmap
/package/AZ-1st-boot/pkginfo
/package/AZ-1st-boot/root/opt/site/AZ-1st-boot.sh
/package/AZ-1st-boot/root/var/svc/manifest/site/AZ-1st-boot.xml
## Validating control scripts.
## Packaging complete.
 
      

Wednesday, September 19, 2012

Non-relocatable SVr4 package

Packages advantages, such as consistency and standardization are quite obvious.
Of course one can get very complex about it, but it suffices to play it simple.
   
An essential distinction is between relocatable and non-relocatable options.
Relocatable packages comprises a directory hierarchy that can be rooted at will.
Non-relocatable packages merge their contents into the preexistent directory hierarchy.
   
It's very easy to create a simple non-relocatable SVr4 package.
Begin by organizing disjoint directories where to put sources and output.
A ZFS file-system proves useful for handling many versions via snapshots and clones.
   
# zfs create -o mountpoint=/pkg data/pkg
# zfs create data/pkg/src
# zfs create data/pkg/out
   
It's a good practice to give a package name a well defined prefix.
Typically an abbreviation of an organization name or other acronym is used.
    
# mkdir /pkg/src/PHXsvc1
    
All the subsequent work will be done inside the source directory.
Actually this isn't required but will simplify matters.
   
# cd /pkg/src/PHXsvc1
    
Once in the source directory, create or put the package contents (files and directories).
Adjust mode and ownership as it should be on the target system after the installation.
    
# ll
total ...
-r--r--r--   1 root  sys    934 Sep 13 15:36 manifest.xml
-r-xr-xr-x   1 root  sys    582 Sep 12 15:55 script.sh
...
   
Generate the base prototype.
The above and below highlights shows related information.
    
# find . ! -name . -a ! -name prototype | pkgproto > prototype
# cat prototype 
f none manifest.xml 0444 root sys
f none script.sh 0555 root sys
   
Edit the prototype to describe where each item will be placed on the target system.
New directories (d) can be created and existent ones can be referenced within new files (f).
Except for JumpStart, when packaging SMF services set the class to manifest.
Every new class that's used must also be referenced in pkginfo's CLASSES.
The below highlights shows what's been edited in the base prototype.
          
# cat prototype 
f manifest /var/svc/manifest/site/manifest.xml 0444 root sys
d none /opt/site 0755 root sys
f none /opt/site/script.sh 0555 root sys
     
Create the information files copyright, pkginfo and scripts in the source directory.
For SMF manifests the search clause is required to point to supporting scripts.
For JumpStart, the search clause and the supporting scripts shouldn't be used.
Reference them all in the prototype along with the package contents.
   
# cat copyright

  Copyright (c) 2012 ...
  All Rights Reserved.


  This product is protected by copyright and distributed under
  licenses restricting copying, distribution, and decompilation.


  
# cat pkginfo
PKG="PHXsvc1"
NAME="Non-relocatable sample package"
ARCH="i386,sparc"
VERSION="1.0"
CATEGORY="system"
DESC="Non-relocatable package demonstration"
VENDOR="AZ - Learnings on Solaris"
PSTAMP="Sep 19, 2012"
CLASSES="none manifest" 
   
# cat prototype 
! search /usr/sadm/install/scripts/
i copyright
i pkginfo
i i.manifest
i r.manifest

f manifest /var/svc/manifest/site/manifest.xml 0444 root sys
d none /opt/site 0755 root sys
f none /opt/site/script.sh 0555 root sys
   
Finally, create the package.    
   
# pkgmk -o -d /pkg/out
## Building pkgmap from package prototype file.
## Processing pkginfo file.
WARNING: missing directory entry for </opt>
WARNING: missing directory entry for </var>
WARNING: missing directory entry for </var/svc>
WARNING: missing directory entry for </var/svc/manifest>
WARNING: missing directory entry for </var/svc/manifest/site>
## Attempting to volumize 3 entries in pkgmap.
part  1 -- 31 blocks, 17 entries
## Packaging one part.
/pkg/out/PHXsvc1/pkgmap
/pkg/out/PHXsvc1/pkginfo
/pkg/out/PHXsvc1/root/opt/site/script.sh
/pkg/out/PHXsvc1/root/var/svc/manifest/site/manifest.xml
/pkg/out/PHXsvc1/install/copyright
/pkg/out/PHXsvc1/install/i.manifest
/pkg/out/PHXsvc1/install/r.manifest
## Validating control scripts.
## Packaging complete.
       
It may be convenient to convert the package to a streamed format.
Notably because it's easier to deal with a single file instead of a directory hierarchy.
     
# pkgtrans /pkg/out /pkg/out/PHXsvc1.pkg PHXsvc1
 
# ll /pkg/out 
total ...
drwxr-xr-x   3 root  root      5 Sep 19 10:28 PHXsvc1
-rw-r--r--   1 root  root   3.5K Sep 19 10:55
PHXsvc1.pkg
...
   

A few verifications could be:
   
# pkgchk -d /pkg/out/PHXsvc1.pkg all 
Checking uninstalled stream format package <PHXsvc1> from </pkg/out/PHXsvc1.pkg>
## Checking control scripts.
## Checking package objects.
## Checking is complete.
  
  
# pkginfo -l -d /pkg/out/PHXsvc1.pkg
   PKGINST:  PHXsvc1
      NAME:  Non-relocatable sample package
  CATEGORY:  system
      ARCH:  i386,sparc
   VERSION:  1.0
    VENDOR:  AZ - Learnings on Solaris
      DESC:  Non-relocatable package demonstration
    PSTAMP:  Sep 19, 2012
    STATUS:  spooled
     FILES:        8 spooled pathnames
                   1 directories
                   1 executables
                   5 package information files
                   3 blocks used (approx)
   
       

Monday, September 10, 2012

Patching the boot miniroot

Assume that the setup is according to Custom JumpStart framework setup.
Consider that no operations have happened yet from /install/Solaris_10/Tools.
The booting miniroot should be unpacked to a subdirectory, not a ZFS dataset.
  
IMPORTANT
It can be considered best practice to patch the boot miniroot as soon as possible.
It's known to be necessary when performing a flash install of an updated ZFS-root image.
/install/Solaris_10/Tools should already exist as per the previous assumptions.
For correct patchadd behavior patch 119255-86 / 119254-86 must be installed.
  
The whole operation is as follows:
  
# mkdir /patch/miniroot
# /boot/solaris/bin/root_archive
    unpackmedia /install /patch/miniroot 

# cp -p /patch/miniroot/sbin/rc2{,.0}
# cp -p /patch/miniroot/sbin/sulogin{,.0}
  
# smpatch download -d /tmp -f -i 147441
147441-23 has been validated.
  
# patchadd -C /patch/miniroot /tmp/147441-23 
Patch 147441-23 has been successfully installed.
See /patch/miniroot/var/sadm/patch/147441-23/log for details
Executing postpatch script...
Patch packages installed:
  SUNWbtool
  SUNWcakr
  SUNWckr
  SUNWcsl
  SUNWcslr
  SUNWcsr
  SUNWcsu
  SUNWesu
  SUNWgss
  SUNWmdb
  SUNWmdbr
  SUNWmptsas
  SUNWnfsckr
  SUNWnfscr
  SUNWnfscu
  SUNWnfsskr
  SUNWnfssu
  SUNWos86r
  SUNWxvmpv
  SUNWzfskr
  SUNWzfsr
  SUNWzfsu
  SUNWzoneu
  
# export SVCCFG_REPOSITORY=/patch/miniroot/etc/svc/repository.db
  
# svccfg -s system/manifest-import setprop start/exec = :true
# svccfg -s system/filesystem/usr setprop start/exec = :true
# svccfg -s system/identity:node setprop start/exec = :true
# svccfg -s system/device/local setprop start/exec = :true
# svccfg -s network/loopback:default setprop start/exec = :true
# svccfg -s network/physical:default setprop start/exec = :true
# svccfg -s milestone/multi-user setprop start/exec = :true

  
# mv /patch/miniroot/sbin/rc2{.0,}
# mv /patch/miniroot/sbin/sulogin{.0,}

# /boot/solaris/bin/root_archive
    packmedia /install /patch/miniroot

Now, other boot and/or JumpStart servers can be more easily updated.
Assuming that the updated server js-01 can (rw) mount js-02:/install:
  
js-01 # cd /install
js-01 # find boot Solaris_10/Tools/Boot |

          cpio -pdum /net/js-02/install
   

Thursday, September 6, 2012

Custom JumpStart flash install

Assume that the setup is according to Custom JumpStart framework setup.
Consider 192.168.0.12 as the IP address of the js-01 X86 Custom JumpStart server.
 
Solaris have always addressed the enterprise needs.
Automated installation is just another example of that.
All that's needed is already included built-in.
 
Solaris is continuously evolving and since Solaris 10 U9, support for Flash Archives have been incorporated to ZFS-root-based Custom JumpStart installations.
 
Compared to the conventional process, the flash install option is simpler.
The main benefit is that a fully updated system can be deployed in one step.
The conventional process requires a time-consuming post-installation update.
Since Solaris has many updates, currently U10 for Solaris 10, the catch-up is long.
Nonetheless, the conventional process is still invaluable to get to consistent images.
 
Another benefit is that it can be part of a disaster recovery procedure for a global zone (GZ).
This assumes, of course, that the non-global zones are backed up by other means.
My preferred NGZ backup option is through ZFS snapshots and streams.

The initial step is, of course, to create a Flash Archive base installation image.
As mentioned, the conventional Custom JumpStart can be used as a starting point.
In fact, I'd say that, for consistency, it's the preferred way to start creating an image.
  
Don't create NGZs for applications yet as they'll be carried out on a later phase.
  
Next, fully update the system.
Begin by applying the most recent patchset for the bulk part.
Wrap up by taking advantage of smpatch for a more fine grained control. 
Use no alternate BEs (ABE) as they aren't supported in the image.
   
Perform some clean-up before creating the image. 
Verify /var/tmp and possibly delete /var/run and inactive BEs (boot environments).
  
Finally, archive the image to where it can be used by Custom JumpStart.
 
js-01:~ # zfs create
            -o mountpoint=/images
            -o sharenfs='rw=base-01,ro=...' ...
js-01:~ # chmod 1777 /images 
 
base-01:~ # mount -F lofs -o nosub / /mnt
base-01:~ # [ -d /mnt/var/run ] && rm -r /mnt/var/run
base-01:~ # ls -la /mnt/var
base-01:~ # umount /mnt
base-01:~ # rm -r /var/tmp/...
base-01:~ # ludelete -R
 
base-01:~ # flarcreate
              -S -n base-01 /net/js-01/images/base-01.flar
 
js-01:~ # chown root:root /images/base-01.flar
  
If not going to use a Flash Archive file anytime soon, consider compressing it.
To achieve the highest compression ratio 7za is the answer.
 
# 7za a base-01{,.flar}
# [ -f base-01.7z ] && rm base-01.flar
  
A few adjustments to the Custom JumpStart profile and rules are needed as follows:
  
# cat rules

# keyword value   begin-script  profile          finish-script
# ------- ------- ------------- ---------------- ---------------
  karch   i86pc   -             profile_x86_flar finish_x86_flar

# cat profile_x86_flar
#
# keyword               value
# --------------------  -------------------------------------
  install_type          flash_install
  archive_location      nfs 192.168.0.12:/images/base-01.flar
  partitioning          explicit
  pool                  rpool auto auto auto mirror any any


# cat finish_x86_flar
#!/bin/sh
  
BASE=/a
BIN=$BASE/usr/bin
SBIN=$BASE/usr/sbin

#................................................................
# X86 fix - The boot device isn't where Solaris was installed
  
$BIN/sync
$SBIN/reboot
   
There's still one important problem: patching the boot miniroot.
Due to the natural evolution of ZFS, the miniroot must be updated accordingly.
The original booting miniroot can't handle the recent ZFS updates on a fully updated image.