GRUB menu

There's a few things we can do to customize the GRUB menu.
In general I tend to keep the defaults as much as possible.
One little thing I always like to touch is the entries' texts.

By default GRUB menu entries get the same name as a corresponding BE.
When I update a system I use specify the new version as the BE name such as:

# pkg update --be-name 11.1-7.5.0
# beadm create -e 11.1-7.5.0 11.1-7.5.0-bk

As a side effect, this generates a corresponding GRUB menu entry such as:

# bootadm list-menu
...
0 Oracle Solaris 11.1
1 11.1-7.5.0
2 11.1-7.5.0-bk
 
This is convenient and expected, but it isn't very cute.
For example, I'd like to prepend the "Oracle Solaris" string.

Of course we can manually edit the appropriate file as in previous releases.
In fact, that used to be the only way to customize anything about GRUB.
Fortunately (or not) we now count on bootadm sucommands:

# bootadm
...
bootadm set-menu ... key=value
bootadm list-menu ... -i
bootadm add-entry ... 
bootadm remove-entry ... -i
bootadm change-entry ... -i {key=value}+ [set-default]
bootadm generate-menu [-f] ...

In particular, for the purpose of this post if suffices to me just the following:

# bootadm change-entry -i 1 title="Oracle Solaris 11.1-7.5.0"
# bootadm change-entry -i 2 title="Oracle Solaris 11.1-7.5.0-bk"
# bootadm list-menu
...
0 Oracle Solaris 11.1
1 Oracle Solaris 11.1-7.5.0
2 Oracle Solaris 11.1-7.5.0-bk

That's really cool, safe and more comfortable than manual editing.
Check bootadm(1M) for more detail.

For example, it's now easy to create boot entries for:

• single user mode/state (kernel argument -s).
• backups on another ZFS pool

    

Shared memory run sample 1.0

Let's follow a shared memory code sample 1.0 on a debugger and watch.
This should clear up any remaining doubts and misunderstandings.
Yet for an additional example, check shared memory code sample 2.0.

The particular environment configuration where the code is run isn't relevant.
I believe it suffices to say that it presents:

• Solaris 11.1 SRU 7.5 running on HP ProLiant BL460c G7.
   
• 24 GiB of physical memory and 18 GiB available virtual memory.
  This is due to some physical memory being reserved to VirtualBox.
  VirtualBox has no other relation to this post, it's there by accident.
  Sure there are VirtualBox guests running and consuming memory.
     
• Disk-based swap is defined at 4 GiB and is not being used at first.
  Certainly, the available virtual memory includes this figure.
     
• The binary will be run under group.staff project resource controls.
  The resource control default was at around 5.99 GiB, ¼ of physical memory.
  Hence, it was adjusted to 20 GiB in order to cause no interferences here.
  

So here is the essential starting point settings and figures:

# tail /etc/system

...

* 24GB - 1.5GB -> 22GB - 5% -> 20GB due to VirtualBox

set zfs:zfs_arc_max=0x400000000

# swap -lh; swap -sh

swapfile                   dev    swaplo   blocks     free

/dev/zvol/dsk/rpool/swap 214,2        4K     4.0G     4.0G

total: 436M allocated + 211M reserved = 648M used, 18G available

# projmod 
  -K "project.max-shm-memory=(privileged,20G,deny)" 
  group.staff

# projects -l group.staff

group.staff

 projid : 10

 comment: ""

 users  : (none)

 groups : (none)

 attribs: project.max-shm-memory=(privileged,21474836480,deny)

---

The 1^st sample run will require 17 GiB of ISM.
As seen this would require enough available lockable physical memory.
This particular run will fail, but note how the swap figures are affected.
Virtual memory isn't directly involved, but it's availability is recomputed.
This is because virtual memory is partially backed by free physical memory.
As physical memory is reserved, less of it is free for backing virtual memory.
This was also discussed in the swap analysis post.

The essential excerpts are as follows:

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G
total: 448M allocated + 212M reserved = 660M used, 18G available

// size == 17 GiB

int id = ::shmget( 1, size, IPC_CREAT | IPC_EXCL | 0660 );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G
total: 448M allocated + 17G reserved = 18G used, 772M available

// Failure!

void * p = ::shmat( id, 0, SHM_SHARE_MMU );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G
total: 448M allocated + 17G reserved = 18G used, 816M available

::shmctl( id, IPC_RMID, 0 );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G
total: 448M allocated + 212M reserved = 660M used, 18G available

---

The 2^nd sample run will also require 17 GiB, this time of DISM kind.
As seen this would entirely rely on virtual memory and directly affect swap.
This particular run will succeed as no physical memory is initially required.

The essential excerpts are as follows:

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 448M allocated + 212M reserved = 660M used, 18G available

 

// size == 17 GiB

int id = ::shmget( 1, size, IPC_CREAT | IPC_EXCL | 0660 );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 448M allocated + 17G reserved = 18G used, 800M available

// Success!

void * p = ::shmat( id, 0, SHM_PAGEABLE );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 448M allocated + 17G reserved = 18G used, 792M available

# ipcs -mA

IPC status ...
T ID KEY ... NATTCH       SEGSZ  CPID ... ISMATTCH     PROJECT
Shared Memory:
m 24 0x1 ...      1 18253611008 10385 ...        1 group.staff

# pmap -x 10385

10385:    /.../shm_v01/dist/Debug/...

         Address    Kbytes       RSS ... Mapped File

0000000000400000         8         8     shm_v01

0000000000411000         4         4     shm_v01

0000000000412000       160        68       [ heap ]

FFFF80FB40000000  17825792         -       [ dism shmid=0x18 ]

...

// Somewhat lengthy run...

::memset( p, '*', size );

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 448M allocated + 17G reserved = 18G used, 792M available

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 8.6G allocated + 9.0G reserved = 18G used, 792M available

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 15G allocated + 2.7G reserved = 18G used, 792M available

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     3.7G

total: 17G allocated + 214M reserved = 18G used, 792M available

# pmap -x 10385

10385:    /.../shm_v01/dist/Debug/...

         Address    Kbytes       RSS ... Mapped File

0000000000400000         8         8     shm_v01

0000000000411000         4         4     shm_v01

0000000000412000       160        68       [ heap ]

FFFF80FB40000000  17825792  17510200       [ dism shmid=0x18 ]

...

switch ( ::shmdt( p ) )

# pmap -x 10385

10385:    /.../shm_v01/dist/Debug/...

         Address    Kbytes       RSS ... Mapped File

0000000000400000         8         8     shm_v01

0000000000411000         4         4     shm_v01

0000000000412000       160        68       [ heap ]

...

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     3.7G

total: 17G allocated + 214M reserved = 18G used, 792M available

switch ( ::shmctl( id, IPC_RMID, 0 ) )

# swap -lh; swap -sh

swapfile             dev    swaplo   blocks     free

/dev/swap             -         4K     4.0G     4.0G

total: 460M allocated + 201M reserved = 664M used, 18G available

From the above output we can list a few interesting conclusions, which I believe certainly will help improve knowledge on shared memory:
    

• ISM relies exclusively on the ability to lock pages of physical memory. It doesn't really participate on virtual memory (virtual swap) although its usage affects the total amount of available virtual memory since any unused physical memory is also used to form virtual memory (virtual swap). As such, disk-based swap space (swap -lh) is never impacted by ISM. At least in theory, ISM can offer better performance as it isn't subject to memory paging.
     
• DISM shares the optimizations of ISM but relies on a completely distinct layer, the virtual memory (virtual swap), not on free physical memory. The main advantages of DISM over ISM is its ability to be larger than the lockable available physical memory and be paged out from memory if it's not being used. So it does can make active use of disk-based swap space (swap -lh). If paging becomes frequent, then performance will certainly suffer.  
     
• By the last output of pmap -x 10385 before ::shmdt(), from the 17825792 Kbytes reserved (= SEGSZ of 18253611008 listed by ipcs -mA), 17510200 KB (or 17930444800 bytes) could fit in physical memory and the rest was accommodated on disk-based swap (18253611008 - 17930444800 = 323166208 ≅ 0.3 GB ). I suspect that the largest lockable available physical memory by that time was that figure of 17510200. In general, I'm not sure if there's an objective method for finding this out.
      
• Although DISM, by default, doesn't lock any memory as ISM, applications can judiciously perform that. Unfortunately, on previous output samples, I have omitted the Locked column of pmap -x output as they were null. When ISM succeeds, the three columns Kbytes, RSS and Locked list the same figure. For DISM, the last two generally vary or float, with Kbytes ≥ RSS ≥ Locked, and which is precisely the advantage of DISM.
    

For very specialized applications such as Oracle Database, I'd say that if the server hardware is dedicated to an instance of it, then DISM would be advantageous with respect to the ability to dynamically adjusting its memory structures up to the defined DISM limit, avoiding application restarts.
 
In terms of resource control, beyond max-shm-memory, there is the max-shm-ids which refers to how many pointers or segments of shared memory can be obtained. What's important to know is that the larger the memory page size, the lesser of them are needed. Thus, SPARC architectures, possessing larger memory pages, tend to consume less of them. In addition, there is the max-locked-memory which affects both ISM and DISM. Remember that DISM may also lock physical memory pages. Of course max-rss, max-swap and max-address-space may also affect limits. I expect to explore some or all of this resource controls later on another post.
  

Shared memory code sample 1.0

A few lines of code may help understanding shared memory, ISM and DISM.
Compile the following code to 64-bits to get beyond the 4 GiB barrier.
In Solaris Studio 12.3 this simply requires compilation option -m64.
 
/*
 * File:   main.cpp
 * Author: AZ
 *
 * Created on May 21, 2013, 3:06 PM
 */

#include <iostream>

#include <sys/ipc.h>
#include <sys/shm.h>

/*
 * This is just a raw demonstration of the shared memory API.
 * To avoid leaks use "resource acquisition is initialization".
 */
  
int main()
{
   //
   // Play with 17 GiB of shared memory!
   //
  
   std::size_t const size = 17UL * 1024UL * 1024UL * 1024UL;

   std::cout << "Getting shared memory. ";

   int id = ::shmget( 1, size, IPC_CREAT | IPC_EXCL | 0660 );

   if ( id == -1 )
   {
      std::cout << "Failure!" << std::endl;
   }
   else
   {
      std::cout << "Success!" << std::endl;

      std::cout << "Attaching to shared memory. ";

      //
      // For a "standard" shared memory segment, 
      // don't pass any flags, that is, pass 0.
      //
      // SHM_RND [Optional]
      // Asks to round down the address to a page boundary.
      // Not necessary for ISM or DISM.
      //
      // SHM_SHARE_MMU
      // Asks for a ISM segment locked in physical memory.
      //
      // SHM_PAGEABLE
      // Asks for a DISM segment pageable (virtual memory).
      //
  
      void * p = ::shmat( id, 0, SHM_SHARE_MMU );

      if ( reinterpret_cast < intptr_t > ( p ) == -1 )
      {
         std::cout << "Failure!" << std::endl;
      }
      else
      {
         std::cout << "Success!" << std::endl;

         std::cout << "Using shared memory." << std::endl;
  
         //         
         // Touch the whole shared segment.
         // Fill it with '*'.
         //
         // This will force the amount reserved
         // to be actually allocated (at least with DISM).
         //

         ::memset( p, '*', size );

         std::cout << "Detaching shared memory. ";

         switch ( ::shmdt( p ) )
         {
            case 0:
               std::cout << "Success!" << std::endl;
               break;

            case -1:
               std::cout << "Failure!" << std::endl;
               break;

            default:
               std::cout << "?" << std::endl;
               break;
         }
      }

      //
      // End of fun!
      //
  
      std::cout << "Removing shared memory. ";

      switch ( ::shmctl( id, IPC_RMID, 0 ) )
      {
         case 0:
            std::cout << "Success!" << std::endl;
            break;

         case -1:
            std::cout << "Failure!" << std::endl;
            break;

         default:
            std::cout << "?" << std::endl;
            break;
      }
   }

   return 0;
}

To take advantage of the above code excerpt for learning, a debugger is required.
Step over the shared memory API calls and watch the CLI ipcs and swap output.
More precisely, the commands are ipcs -mA and swap -sh and swap -lh.

By the way, why do I chose to play with 17 GiB?
That's because my particular test system had 18 GiB to offer.
For more detail showing some real output check shared memory run sample 1.0.
In time I have also added a DISM version with manual locking.
  

Shared memory, ISM and DISM

This topic is somewhat mysterious at first glance.
Furthermore, it seems lacking sufficient documentation.
After a quite long time frustrated with this I finally decided to discover the truth.
As always, this kind of things require us stepping into developer's land.
I believe that a good system administration implies a development.

Shared memory is an efficient way of passing data between processes.
That's part of a broader topic: Interprocess Communication (IPC).

In Unix world there are many standards and versions.
For example, there's the Single Unix Specification.
For this post, please, consider just the following:
  

• <sys/ipc.h>
  <sys/shm.h>
     
• ::shmget() - reserves shared memory.
  ::shmat()  - attaches to shared memory.
  ::shmdt()  - detaches from shared memory.
  ::shmctl() - free shared memory resources.

  
ISM and DISM are Solaris specific shared memory optimizations:

• Intimate Shared Memory (ISM).
  This kind of shared memory is automatically locked by kernel upon segment creation. The locking prevents paging out and a fast I/O locking within the segment, saving significant CPU time. Furthermore, in contrast to "standard" shared memory, kernel V2P (virtual-to-physical) memory address translation structures are shared between processes, which saves kernel memory and also CPU time. In addition, large memory pages are automatically allocated for ISM segments, which reduces the number of memory pointers, thus reducing complexity and resulting in performance improvements, specially on large memory systems. Larger pages, less pointers and less shm-ids. As ISM is always locked, it isn't necessary to provide swap for back it up, which also saves disk space. To obtain an ISM segment, sufficient unlocked physical memory must be available. ISM segments are obtained by passing the SHM_SHARE_MMU flag to shmat(2).
    
• Dynamic ISM (DISM).
  Builds on ISM but in contrast,  it's dynamically changeable. What isn't always clearly stated is that a DISM segment can't be shrunk. The "secret" is to obtain a large DISM segment right from the start. The size of the segment can be as large as the available virtual memory, hence disk swap is always involved. As such DISM reestablishes shared memory as pageable, although better optimized. DISM isn't automatically locked by the kernel. It's the responsibility of the application to lock, mlock(3C), and unlock, munlock(3C), segment pages, according to its own requirements, thus providing dynamic adjustment. DISM segments are obtained by passing the SHM_PAGEABLE flag to shmat(2).

In administering Solaris memory resources, beyond physical memory and virtual memory, depending on the application, it may be important to understand shared memory and locked memory. All these concepts are typically used by RDBMS such as Oracle Database, as an efficient interprocess communication mechanism, such as Oracle's SGA implementation.

For system administrators the more important conclusions on ISM and DISM are:

• ISM doesn't affect swap and requires available physical memory, whereas DISM to be useful, generally relies on swap (not on physical memory). If swapping isn't desired, force ISM in order to not waste disk space for backing shared memory.
    

• ISM and DISM are more efficient than the "standard" shared memory, consisting on a specific Solaris advantage.
    

One final crucial note is that shared memory (of any kind) in latest Solaris 10 and Solaris 11 systems is controlled by the max-shm-memory resource control, which defaults to ¼ of the installed physical memory.

And for wrapping up this subject, here's a shared memory code sample 1.0 and a shared memory run sample 1.0 for a more concrete approach.

A few incomplete references:
  

• Solaris Internals: Solaris 10 and OpenSolaris Kernel Architecture, Ed. 2.
  Richard McDougall and Jim Mauro
  Part II, section 4.4.
    
• Advanced Programming in the UNIX Environment, Ed. 3.
  W. Richard Stevens and Stephen A. Rago.
  Sections 15.6 XSI IPC and 15.9 Shared Memory.
     
• Many My Oracle Support (MOS) documents.
    
• man pages such as shmget(2), shmop(2)and shmctl(2).

   

GRUB password

Nowadays I'd say it's hard to believe that anything is really secure.
Nevertheless one can keep going adding more and more barriers.
The idea is not to be selected as an easier path to attack.
But again, if someone is determined who can tell...

Despite this gave introduction, my goal is to repeat one known tiny bit:
Help preventing unauthorized GRUB configuration change by adding a password.
The method below isn't for GRUB2 (the next generation), but for the older version.

Locate the grub menu file where to configure the password:

# bootadm list-menu
the location ... is: /rpool/boot/grub/menu.lst
default 4
timeout 15
...

Invoke the grub binary to create the password.
Take note of the resulting encrypyted hash.

# /boot/grub/bin/grub
 
    GNU GRUB  version 0.97  (640K lower / 65536K upper memory)
 [ ...
   ...
   ... ]

grub> md5crypt

Password: ***************
Encrypted: $1$...

grub> quit

Edit the grub menu file and include the generated password hash as shown below:

# head -7 /rpool/boot/grub/menu.lst
splashimage /boot/grub/splash.xpm.gz
foreground 343434
background F7FbFF
default 4
timeout 15
password --md5 $1$...
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
...
 
That's all what's need for GRUB1.
For GRUB2 I'm still trying to learn how to do it.