The AMP stack (later)

So it's been verified that for the latests components on an AMP stack, one would have to build the AMP stack by hand. Instead of getting external packages or source code for the very latests versions let's try first to get the best from what's already available in the standard Solaris 11.3 release repository.

The "outdated" meta/group package for the AMP stack is as follows:

# pkg contents -r -o fmri -t depend 'group/feature/amp' 
FMRI
database/mysql-55
web/php-53
web/php-53/extension/php-apc
web/php-53/extension/php-mysql
web/server/apache-22
web/server/apache-22/module/apache-dtrace
web/server/apache-22/module/apache-fcgid
web/server/apache-22/module/apache-php53
 
It's known that the best (latest) main AMP stack components (IPS packages) already/publicly available (on the Solaris 11.3 release repository) are:
 

• apache-24
• mysql-56
• php-56

Hence I'll start by inspecting if among all the dependent packages of each of these 3 "more updated" main packages one can find the corresponding ones required by the original ("outdated") meta/group package, that is, I'll look for:
 

• apache-24/module/apache-dtrace
• apache-24/module/apache-fcgid
• apache-24/module/apache-php56
• php-56/extension/php-apc
• php-56/extension/php-mysql

But as you can see below it seems things can be a bit different with respect to PHP (php-56). I do not yet know if there are any substitutes for the missing packages' functionalities:

$ pkg search -o pkg.name '*:set:pkg.fmri:apache-24/module/*'
PKG.NAME
web/server/apache-24/module/apache-dbd
web/server/apache-24/module/apache-dtrace
web/server/apache-24/module/apache-fcgid
web/server/apache-24/module/apache-gss
web/server/apache-24/module/apache-jk
web/server/apache-24/module/apache-ldap
web/server/apache-24/module/apache-lua
web/server/apache-24/module/apache-security
web/server/apache-24/module/apache-ssl
web/server/apache-24/module/apache-ssl-fips-140
web/server/apache-24/module/apache-wsgi-26
web/server/apache-24/module/apache-wsgi-27
web/server/apache-24/module/apache-wsgi-34

$ pkg search -o pkg.name '*:set:pkg.fmri:php-56/extension/*'
PKG.NAME
web/php-56/extension/php-suhosin-extension
web/php-56/extension/php-xdebug

Fortunately, for PHP (php-56) it seems there's a nice solution right out-of-the-box. I've got to know about it when browsing the php-53 php(1) man page which says:

The PHP packages included in Solaris contain many PHP extensions. In order to use these extensions, you must enable them in a PHP configuration file.

Starting with PHP 5.6, Solaris provides a file "conf.d/extensions.ini" under the appropriate /etc/php subdirectory. You may uncomment lines in this file to enable the desired extensions.

To use PHP as an Apache web server module, you must configure Apache appropriately. Starting with PHP 5.6, Solaris provides a sample supplemental configuration file at /etc/apache2/2.4/samples-conf.d/php5.6.conf. To use it without modification, copy it into /etc/apache2/2.4/conf.d/.

The web/server/apache-22/module/apache-php53 package provides PHP 5.3 as a server module for Apache 2.2.

For PHP 5.6, an additional package is not required; an Apache 2.4 server module is included in the web/php-56 package.

Apache 2.2 support is not provided with PHP 5.6 in Solaris.

Now addressing the issue of setting up the communication between PHP 5.6 and MySQL 5.6, by browsing the man page of mysql-55 which is installed at /usr/mysql/5.5/man (although not automatically integrated into the man system), man -M /usr/mysql/5.5/man -s 1 mysqlman, I've go to know about where to find more information about MySQL: http://dev.mysql.com/doc/mysql/en, in fact, https://dev.mysql.com/doc/refman/5.6/en/ (for MySQL in general) and https://dev.mysql.com/doc/apis-php/en/ (for MySQL x PHP specifics). And from all this documentation, the good news seems to be that no additional setup is required to have them communicating nice and smoothly (but I intend to verify that later, of course).

Fortunately, it seems I'm pretty done with all preliminary checking for building my custom AMP stack from the best (latest) main components (IPS packages) already/publicly available (on the Solaris 11.3 release repository).

The next steps (on subsequent posts) are the actual setup of:
 

• Apache 2.4
• MySQL 5.6
• PHP 5.6

PS:

Be happy because I've just got to know that it's possible to download the latest version of MySQL (5.7.18) from https://dev.mysql.com/downloads as long as you have an Oracle account (a matter of just registering). And even better is the fact that there's an option for downloading it as an IPS package (mysql-5.7.18-solaris11-x86_64.pkg.gz).

And chances seems to be that even if Oracle do not update its repository the other components can (with additional efforts) be independently updated/upgraded from the web as well.

   

The AMP stack

I woke up one of these days wondering how good it could be the current status of the well-known AMP (Apache + MySQL + PHP) stack in Solaris 11.3 public (general availability) release repo(sitory). I wished it could shine but I soon realized I shouldn't expect something fascinating because this software bundle isn't mainstream in the Solaris platform and isn't coming from a support repository which is subject to more frequent updates or upgrades.

Anyway someone had to do the dirty work ;-) of checking the current status as frankly it's not so easy to find it out there on the Internet by simply querying Google or YouTube and so on...

So I started on a fresh Solaris 11.3 (release) text-install, at the time of this writing, running on the latest (5.1.18r114002) VirtualBox hosted at another Solaris 11.3 on x86-64, of course :-D

I started by searching the available packages on my release repo(sitory). But the "perfect" search was hard to construct as the man pages pkg(1) and pkg(5), although very descriptive, are rather verbose and difficult to understand in terms of a more practical day by day usage. Nevertheless, after a more thorough and time consuming reading I've managed to digest it and narrow it down to the following:

$ pkg search -o pkg.shortfmri '*:set:pkg.fmri:amp*'
PKG.SHORTFMRI
pkg:/amp@0.5.11-0.133
pkg:/group/feature/amp@0.5.11-0.175.3.0.0.30.0
pkg:/web/amp@0.5.11-0.174.0.0.0.0.0

Then I focused on the highest version available to me:

$ pkg list -a group/feature/amp
NAME (PUBLISHER)   VERSION                    IFO
group/feature/amp  0.5.11-0.175.3.0.0.30.0    ---
 
$ pkg info -r group/feature/amp
          Name: group/feature/amp
       Summary: AMP (Apache, MySQL, PHP) Deployment Kit ...
   Description: ... deployment of an AMP (...) stack ...
      Category: Meta Packages/Group Packages ...
                Web Services/Application and Web Servers (...)
         State: Not installed
     Publisher: solaris
       Version: 0.5.11
 Build Release: 5.11
        Branch: 0.175.3.0.0.30.0
Packaging Date: August 21, 2015 04:52:02 PM
          Size: 5.46 kB
          FMRI: ..:20150821T165202Z

By the way, although the very first command above is much more powerful (and flexible) for searches, in this particular case, the results of the two commands right above seems to be combined into a somewhat equivalent result from the simpler command:

$ pkg list -af amp
NAME (PUBLISHER)   VERSION                    IFO
amp                0.5.11-0.133               --r
group/feature/amp  0.5.11-0.175.3.0.0.30.0    ---
web/amp            0.5.11-0.174.0.0.0.0.0     --r

NOTE

If you were looking for some other "unknown" package, for instance, individually searching for apache, mysql or php packages, I believe you would appreciate the very first form of the package search command I've managed to construct.

Finally, let's proceed to the installation of the bundle (meta/group package) into a new boot environment (good practice) and inspect some of the results:

# beadm list
BE           Flags Mountpoint Space Policy Created
--           ----- ---------- ----- ------ -------
Solaris-11.3 NR    /          3.59G static 2017-04-09 01:59
solaris      -     -          3.87M static 2017-04-08 07:15 
  
# pkg install --be-name Solaris-11.3-AMP group/feature/amp
           Packages to install:  18
           Mediators to change:   1
       Create boot environment: Yes
Create backup boot environment:  No

DOWNLOAD           PKGS         FILES    XFER (MB)   SPEED
Completed         18/18     1091/1091  102.9/102.9  4.1M/s

PHASE                                          ITEMS
Installing new actions                     1446/1446
Updating package state database                 Done
Updating package cache                           0/0
Updating image state                            Done
Creating fast lookup database                   Done
Updating package cache                           1/1

A clone of Solaris-11.3 exists and has been updated and activated.
 
On the next boot the Boot Environment Solaris-11.3-AMP will be
mounted on '/'.  Reboot when ready to switch to this updated BE.

Updating package cache                           1/1

# beadm list
BE               Flags Mountpoint Space Policy Created
--               ----- ---------- ----- ------ -------
Solaris-11.3     N     /          3.20M static 2017-04-09 01:59
Solaris-11.3-AMP R     -          4.32G static 2017-04-11 11:09
solaris          -     -          3.87M static 2017-04-08 07:15

Watching the installation it seems that the meta/group package didn't pull-in the latest versions of Apache, MySQL and PHP available in the repository after all, but I'll have to confirm that in the sequence (yes, perhaps I should have inspected the meta/group package itself before taking any other action, but I'm going slow on the learning curve and having fun).

# init 6
...

# beadm list
BE               Flags Mountpoint Space Policy Created
--               ----- ---------- ----- ------ -------
Solaris-11.3     -     -          6.79M static 2017-04-09 01:59
Solaris-11.3-AMP NR    /          4.42G static 2017-04-11 11:09
solaris          -     -          3.87M static 2017-04-08 07:15

Now let's check what was actually installed on the system:

$ pkg list -a 
  `pkg search -H -o pkg.shortfmri '*:set:pkg.fmri:apache-2?'`
NAME (PUBLISHER)       VERSION                    IFO
web/server/apache-22   2.2.31-0.175.3.1.0.3.0     i--
web/server/apache-24   2.4.12-0.175.3.0.0.30.0    ---

$ pkg list -a 
  `pkg search -H -o pkg.shortfmri '*:set:pkg.fmri:mysql-5?'`
NAME (PUBLISHER)       VERSION                    IFO
database/mysql-50      5.0.91-0.171               --o
database/mysql-51      5.1.37-0.175.3.0.0.30.0    ---
database/mysql-55      5.5.43-0.175.3.0.0.30.0    i--
database/mysql-56      5.6.21-0.175.3.0.0.30.0    ---

$ pkg list -a 
  `pkg search -H -o pkg.shortfmri '*:set:pkg.fmri:php-5?'`
NAME (PUBLISHER)       VERSION                    IFO
web/php-52             5.2.17-0.175.3.0.0.25.0    --o
web/php-53             5.3.29-0.175.3.1.0.2.0     i--
web/php-56             5.6.8-0.175.3.0.0.30.0     ---

Et voilà! That's Solaris! Almost always conservative :-D
Not the oldest, not the latest... Perhaps, just one before the latest...

All right, all right, :-D here's (at least I think so for the moment) the shortest, easiest, fastest, cleanest, finest, more clever way in finding out the actual packages that are pulled-in by the meta/group package:

# pkg contents -r -o fmri -t depend 'group/feature/amp' 
FMRI
database/mysql-55
web/php-53
web/php-53/extension/php-apc
web/php-53/extension/php-mysql
web/server/apache-22
web/server/apache-22/module/apache-dtrace
web/server/apache-22/module/apache-fcgid
web/server/apache-22/module/apache-php53
 
The next steps (on subsequent posts) are the actual setup of:
 

• Apache 2.2
• MySQL 5.5
• PHP 5.3

Bottom line, for the latests, I'll have to build the AMP stack on my own, but in spite of that, Solaris 11.3 also individually offers newer versions of each of these packages so that I can have an overall later version of this stack.
  

The physical memory

The physical memory is a crucial and precious resource and is commonly one of the major system bottlenecks as well as one of the system components that bumps a system price to the skies.

Knowing and, better yet, determining at runtime the amount of physical memory that is physically installed on a host and that is actually available to the system is important to many deployment and administration strategies.

Without recurring to programming at the system APIs level, it is possible to easily determine such figures as shown below.

$ prtconf | grep Mem
Memory size: 8192 Megabytes

# echo ::memstat | mdb -k | grep Total
Total            2096958            7.9G

$ kstat -p -n system_pages | egrep 'avail|physmem|locked|total'
unix:0:system_pages:availrmem    930155
unix:0:system_pages:pageslocked 1162706
unix:0:system_pages:pagestotal  2092861
unix:0:system_pages:physmem     2092861

Note that pagestotal = availrmem + pageslocked and that it seems that interestingly pagestotal = physmem, all in multiples of page sizes.

$ pagesize
4096

Then we can now compare things and better grasp the reality:

$ echo "(2092861 * `pagesize`) / 1024 ^ 2" | bc
8175

$ echo "(2096958 * `pagesize`) / 1024 ^ 2" | bc
8191
 
To me the 16 MB (4096 pages) difference between 8191 and 8175 seems to be fixed (non-pageable) and is still a mystery, a matter to open investigation, perhaps some part of the kernel known only by the internal staff.

That is, according to system's best result it actually sees 8191 MB, 1 MB less than what's physically installed on the host and that's not so hard to wonder why (perhaps set aside for the on-board video or so). Using closer to perfect figures ought to provide more exact results for planning and assessments.
 

Kernel zones & ZFS ARC

Assuming your system meet sufficient kernel zones support requirements one important tunning is the adjustment of the ZFS ARC maximum bytes (the so known zfs_arc_max in /etc/system). I've done a somewhat similar tunning a couple of years ago as tunning best practice right after installing VirtualBox. For kernel zones it may not be just a case of simple best practice but more likely a be advised or neglect it at your own risk!

By the way, according to more recent Solaris public documentation, the host system sees kernel zones just as another application. The required tuning on the host system should take into account all the kernel zones and processes that are anticipated to run on the system.

In the past, for figuring out the current zfs_arc_max I just relied on the c_max bytes from kstat -n arcstats. But more recently Solaris 11.2 documentation refers to ::memstat from mdb -k. So let's just put them in perspective (remembering that other figures from arcstats may play a role not being considered below):

# kstat -n arcstats | grep c_max
    c_max                           7498616832

# echo ::memstat | mdb -k
Page Summary                 Pages             Bytes  %Tot
----------------- ----------------  ----------------  ----
Kernel                      293573              1.1G   14%
ZFS Metadata                 28199            110.1M    1%
ZFS File Data               517332              1.9G   25%
Anon                        269994              1.0G   13%
Exec and libs                 6008             23.4M    0%
Page cache                  328957              1.2G   16%
Free (cachelist)              3779             14.7M    0%
Free (freelist)             628887              2.3G   30%
Total                      2096958              7.9G

# pagesize
4096

To quote the Solaris 11.2 documentation topic:

The suggested value is one-half of what you would like the host ZFS resources to use. For example, if you want ZFS to use less than 2 GB of memory, set the ARC cache to 1 GB, or 0x40000000.

Furthermore the Solaris 11.2 documentation on zfs_arc_max says:

75% of memory on systems with less than 4 GB of memory.
physmem minus 1 GB on systems with greater than 4 GB of memory.

If a future memory requirement is significantly large and well defined, you might consider reducing the value of this parameter to cap the ARC so that it does not compete with the memory requirement. For example, if you know that a future workload requires 20% of memory, it makes sense to cap the ARC such that it does not consume more than the remaining 80% of memory.

But in Solaris 11.3 things start to change a bit. There's a new tunable called user_reserve_hint_pct (from 0 to 99, defaulting to 0, also set in /etc/system as set user_reserve_hint_pct=...) intended to supersede zfs_arc_max.  About the new tunable, Solaris 11.3 documentation says:

Informs the system about how much memory is reserved for application use, and therefore limits how much memory can be used by the ZFS ARC cache as the cache increases over time.

By means of this parameter, administrators can maintain a large reserve of available free memory for future application demands. The user_reserve_hint_pct parameter is intended to be used in place of the zfs_arc_max parameter to restrict the growth of the ZFS ARC cache.

If a dedicated system is used to run a set of applications with a known memory footprint, set the parameter to the value of that footprint.

For upward adjustments, increase the value if the initial value is determined to be insufficient over time for application requirements, or if application demand increases on the system. Perform this adjustment only within a scheduled system maintenance window. After you have changed the value, reboot the system.

For downward adjustments, decrease the value if allowed by application requirements. Make sure to use decrease the value only by small amounts, no greater than 5% at a time.

  

Kernel zones support

The advent of kernel zones in Solaris 11.2 is another great improvement to Solaris. But it may not be supported on aging hardware as I may have just found out. I happen to use a box more than 5 years old, from later 2009, which seems not support all the required virtualization technology for kernel zones. But I'm pending confirmation if my issue is just because I have VirtualBox installed on my x86-64 and this is posing some sort of conflict with kernel zones availability in terms of lack of (already allocated to VirtualBox) virtualization resources.

So if you're planning to set aside some "cool hardware" for your Solaris 11.3 kernel zones, I suggest you learn from this experience of mine beforehand in order to make sure your "cool hardware" and system setup meet all the requirements.

You may start by checking the man page solaris-kz(5):

...

The  solaris-kz  brand  uses certain hardware features which may not be available in older systems, or in virtualized environments.  To  detect whether  a  system  supports  the  solaris-kz brand, install the brand-solaris-kz package and then run the virtinfo command.

# virtinfo -c supported list kernel-zone

If kernel-zone is not shown in the supported list, you can  see  syslog for  more information. Messages pertaining to kernel zones will contain the string kernel-zone.
 
...

In my case, in general, I've got:

$ virtinfo -c supported list kernel-zone
kernel-zone: no such supported virtual environment found

$ virtinfo
NAME            CLASS    
non-global-zone supported

And under VirtualBox 5.1.18 r114002, I've got:

$ virtinfo
NAME            CLASS    
virtualbox      current  
non-global-zone supported

Well, it's true that in the logs you'll have to look for kernel-zone.
But you'll have to do so in /var/adm/messages instead.
So I set out to further investigate what was missing.

For my physical box I've got:

$ grep kernel-zone /var/adm/messages | cut -d: -f5,6 | sort -u
... environment not supported: VMX already in use
... unsupported Intel model 15

And under VirtualBox (on that same physical box), I've got:

$ grep kernel-zone /var/adm/messages | cut -d: -f5,6 | sort -u
... environment not supported: CPU doesn't have VMX

According to a wikipedia article on x86 virtualization, VMX happen to be the designation for the CPU flag related to VT-x support. What caught my attention was the single message VMX already in use. It appeared just once and it's true I have enabled virtualization support on my physical box's BIOS which makes me wonder if the situation would change in favor of kernel-zones meeting all its requirements if I completely uninstall VirtualBox. I haven't tried it yet nor I'm willing to do it right now as I do heavy use of VirtualBox. But depending on the scenario, the trade-off would certainly pay off.

By the way, I'd like to mention that I did try something less drastic than uninstalling VirtualBox. I tried disabling (setting to off) the VirtualBox's property hwvirtexclusive but that didn't make any difference in solving the problem (at least as to version 5.1.18). Later I found a forum entry about this that claims to have worked, but this was for earlier versions.