The physical memory is a crucial and precious resource and is commonly one of the major system bottlenecks as well as one of the system components that bumps a system price to the skies.
Knowing and, better yet, determining at runtime the amount of physical memory that is physically installed on a host and that is actually available to the system is important to many deployment and administration strategies.
Without recurring to programming at the system APIs level, it is possible to easily determine such figures as shown below.
$ prtconf | grep Mem
Memory size: 8192 Megabytes
# echo ::memstat | mdb -k | grep Total
Total 2096958 7.9G
$ kstat -p -n system_pages | egrep 'avail|physmem|locked|total'
unix:0:system_pages:availrmem 930155
unix:0:system_pages:pageslocked 1162706
unix:0:system_pages:pagestotal 2092861
unix:0:system_pages:physmem 2092861
Note that pagestotal = availrmem + pageslocked and that it seems that interestingly pagestotal = physmem, all in multiples of page sizes.
$ pagesize
4096
Then we can now compare things and better grasp the reality:
$ echo "(2092861 * `pagesize`) / 1024 ^ 2" | bc
8175
$ echo "(2096958 * `pagesize`) / 1024 ^ 2" | bc
8191
To me the 16 MB (4096 pages) difference between 8191 and 8175 seems to be fixed (non-pageable) and is still a mystery, a matter to open investigation, perhaps some part of the kernel known only by the internal staff.
That is, according to system's best result it actually sees 8191 MB, 1 MB less than what's physically installed on the host and that's not so hard to wonder why (perhaps set aside for the on-board video or so). Using closer to perfect figures ought to provide more exact results for planning and assessments.
Showing posts with label Swap. Show all posts
Showing posts with label Swap. Show all posts
Monday, April 10, 2017
Wednesday, July 2, 2014
Limiting /tmp size
It's no doubt that the /tmp file system is very useful and an indispensable item.
But as you know it's based on the system provided swap space.
Thus it's a good idea to limit /tmp.
This is easily done by edint /etc/vfstab.
But I reboot is required in order to changes take effect.
Here's an example of limiting /tmp to 2 GB.
Just edit the corresponding line as below:
$ grep tmp /etc/vfstab
swap - /tmp tmpfs - yes size=2048m
NOTE
As an example of failing to identify a baseline before setting a limit, take my experience on setting a 1 GB limit and not noticing /tmp filling up until I get an error message from X screensaver after locking my desktop stating "ftruncate() error: no space left on device". I couldn't find any other indication on /var/adm/messages, SMF services or ZFS pools and file systems that could indicate a device filling up, until I realize the /tmp limit just because I was researching the possibility of encrypting swap, which, by the way, is still unsupported in Solaris 11.1. In fact, in the event of those "mysterious" error messages only DTrace can help spot the source or at least provide a good clue.
But as you know it's based on the system provided swap space.
Thus it's a good idea to limit /tmp.
This is easily done by edint /etc/vfstab.
But I reboot is required in order to changes take effect.
Here's an example of limiting /tmp to 2 GB.
Just edit the corresponding line as below:
$ grep tmp /etc/vfstab
swap - /tmp tmpfs - yes size=2048m
NOTE
But be aware of one caveat when setting a limit: don't set it too low for a normal system operation, that is, before setting a value, take a baseline of the typical space consumption of /tmp and set a reasonable value above it. Otherwise, you'll likely see unexpected error messages in apparently non-related components or subsystems that fail to allocate some space in /tmp.
As an example of failing to identify a baseline before setting a limit, take my experience on setting a 1 GB limit and not noticing /tmp filling up until I get an error message from X screensaver after locking my desktop stating "ftruncate() error: no space left on device". I couldn't find any other indication on /var/adm/messages, SMF services or ZFS pools and file systems that could indicate a device filling up, until I realize the /tmp limit just because I was researching the possibility of encrypting swap, which, by the way, is still unsupported in Solaris 11.1. In fact, in the event of those "mysterious" error messages only DTrace can help spot the source or at least provide a good clue.
Friday, July 12, 2013
MPSS & large chunks of memory
Solaris as a high-end Unix can handle lots of memory.
I just say large for something that other OS might consider huge.
Nevertheless, a good application ought to cooperate with the system.
One way of coping with that is obtaining specialized chunks of memory.
This strategy can be an important overall optimization.
Solaris is not only a high capacity system but also a very flexible one.
In fact, it has to be, otherwise it wouldn't sustain an almost linear scalability.
Examples of that, related to main memory, are (D)ISM and MPSS.
It seems very obvious to allocate large chunks of memory for specialized purposes. The problem is implementing it properly. Part of the challenge is knowing what the underlying system provides, in this case, hopefully, Solaris. For instance, it's not difficult in Solaris to obtain a large chunk of locked memory, that's a multiple of the largest supported virtual memory page size, which is always properly aligned to the strictest C++ data type of the platform.
inline bool hat_advise_bssbrk( std::size_t const pagesize )
{
::memcntl_mha mha;
mha.mha_cmd = MHA_MAPSIZE_BSSBRK;
mha.mha_flags = 0;
mha.mha_pagesize = pagesize;
return
::memcntl
(
0,
0,
MC_HAT_ADVISE,
reinterpret_cast< caddr_t >( & mha ),
0,
0
)
== 0;
}
// A friendly usage pattern.
// Doesn't handle any exceptions.
void f()
{
// Chunk size should be a large-page size multiple
std::size_t const pagesize = largest_pagesize();
std::size_t const size = 512 * pagesize;
// Advise HAT to adopt a large-page size
// May trigger fix-up viewable with pmap -xs
// So from now on the heap will use large-page sizes
if ( !hat_advise_bssbrk( pagesize ) )
::perror( NULL );
// Friendly reserve a large-page multiple (chunk) from heap
// May trigger additional fix-up viewable with pmap -xs
// Reserved address range will also be viewable
void * p = ::memalign( pagesize, size );
// Optional, but useful to prevent swapping
if ( ::mlock( p, size ) != 0 )
::perror( NULL );
// Touch the reserved memory
// Triggers the actual allocation
::memset( p, '*', size );
if ( ::munlock( p, size ) != 0 )
::perror( NULL );
// Done
::free( p );
}
Thanks again to Solaris the solution is easy.
Consider the following slight variations from the previous code:
inline bool hat_advise_va
(
void const * const p,
std::size_t const size,
std::size_t const pagesize
)
{
::memcntl_mha mha;
mha.mha_cmd = MHA_MAPSIZE_VA;
mha.mha_flags = 0;
mha.mha_pagesize = pagesize;
return
::memcntl
(
static_cast< caddr_t >
( const_cast< void * >( p ) ),
size,
MC_HAT_ADVISE,
reinterpret_cast< caddr_t >( & mha ),
0,
0
)
== 0;
}
// A typical usage pattern.
// Doesn't handle any exceptions.
void f()
{
// Chunk size should be a large-page size multiple
std::size_t const pagesize = largest_pagesize();
std::size_t const size = 512 * pagesize;
// Friendly reserve a large-page multiple (chunk) from heap
// May trigger additional fix-up viewable with pmap -xs
// Reserved address range will also be viewable
void * p = ::memalign( pagesize, size );
// Advise HAT to adopt a large-page size for the chunk
// May trigger fix-up viewable with pmap -xs
if ( !hat_advise_va( p, size, pagesize ) )
::perror( NULL );
// Optional, but useful to prevent swapping
if ( ::mlock( p, size ) != 0 )
::perror( NULL );
// Touch the reserved memory
// Triggers the actual allocation
::memset( p, '*', size );
// May immediately page-out parts of chunk
if ( ::munlock( p, size ) != 0 )
::perror( NULL );
// Done
::free( p );
}
By the way, if I want to check the page size backing a certain region:
inline std::size_t pagesize( void const * const p )
{
uint_t const request = MEMINFO_VPAGESIZE;
uint64_t output;
uint_t validity;
if ( ::meminfo
(
reinterpret_cast< uint64_t const * >( & p ),
1,
& request,
1,
& output,
& validity
)
== 0
)
// Is p a valid virtual address?
if ( validity & 1 )
// Has the virtual address been touched?
// Is there any memory page backing it?
if ( validity & 2 )
return output;
// No page is backing the virtual address
return 0;
}
For exemplifying the behavior of the code compiled (-g -m64) on an Intel x64, from a debugging session, let's take excerpts of the heap life-cycle from a series of:
$ pmap -xs `pgrep <program>` | head -25
The program's heap starts as follows:
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
After the call to ::memalign():
0000000000438000 1820 - - - - rw--- [ heap ]
00000000005FF000 4 4 4 - 4K rw--- [ heap ]
0000000000600000 1048576 - - - - rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
0000000040601000 216 - - - - rw--- [ heap ]
0000000040637000 4 4 4 - 4K rw--- [ heap ]
FFFF80FFB8EB0000 4 4 - - 4K r-x-- ...
...
After the call to ::mlock():
0000000000600000 1048576 1048576 - 1048576 - rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
After the call to ::memset():
0000000000600000 1048576 1048576 1048576 1048576 2M rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
After the call to ::munlock():
0000000022C00000 8192 8192 - - - rw--- [ heap ]
0000000023400000 477184 477184 477184 - 2M rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
Among other things, note that the next available heap address lies on a default page size, in the case of this system, a 4K page, instead of the largest page size, again, in the case of this system, a 2M page. This is exactly the fine-grained manual and dynamic control I was looking for, thanks to Solaris, of course!
I just say large for something that other OS might consider huge.
Nevertheless, a good application ought to cooperate with the system.
One way of coping with that is obtaining specialized chunks of memory.
This strategy can be an important overall optimization.
Solaris is not only a high capacity system but also a very flexible one.
In fact, it has to be, otherwise it wouldn't sustain an almost linear scalability.
Examples of that, related to main memory, are (D)ISM and MPSS.
It seems very obvious to allocate large chunks of memory for specialized purposes. The problem is implementing it properly. Part of the challenge is knowing what the underlying system provides, in this case, hopefully, Solaris. For instance, it's not difficult in Solaris to obtain a large chunk of locked memory, that's a multiple of the largest supported virtual memory page size, which is always properly aligned to the strictest C++ data type of the platform.
ALTERNATIVE 1
Allocating an ISM segment very conveniently and automatically addresses this strategy as it might have been partially revealed on ISM run sample 3.0. Note that for the ISM specific case the respective sample code didn't need any explicit locking control, that is, it didn't require the proc_lock_memory privilege, neither the mlock() / munlock() system calls.ALTERNATIVE 2
What's also very cool in Solaris is that you can also take advantage of this strategy for the program's heap, by manually and dynamically:- Reconfiguring the virtual memory page size that backs up the program's heap to the platform's largest supported value;
- Allocating a large chunk of memory that is aligned to and is a multiple of the platform's largest supported virtual memory page size;
- Optionally, locking all the large virtual memory pages comprising the chunk so that no paging ever happens. This consumes more main memory but may improve performance and avoid the corresponding swap space reservation.
template< typename T >
inline T * tmp_array( std::size_t const n )
{
return static_cast< T * >( ::alloca( n * sizeof( T ) ) );
}
std::size_t largest_pagesize()
{
std::size_t largest = ::sysconf( _SC_PAGESIZE );
int n = ::getpagesizes( NULL, 0 );
std::size_t * size = tmp_array< std::size_t >( n );
if ( ::getpagesizes( size, n ) != -1 )
while ( --n >= 0 )
if ( size[ n ] > largest )
largest = size[ n ];
return largest;
}
inline T * tmp_array( std::size_t const n )
{
return static_cast< T * >( ::alloca( n * sizeof( T ) ) );
}
std::size_t largest_pagesize()
{
std::size_t largest = ::sysconf( _SC_PAGESIZE );
int n = ::getpagesizes( NULL, 0 );
std::size_t * size = tmp_array< std::size_t >( n );
if ( ::getpagesizes( size, n ) != -1 )
while ( --n >= 0 )
if ( size[ n ] > largest )
largest = size[ n ];
return largest;
}
inline bool hat_advise_bssbrk( std::size_t const pagesize )
{
::memcntl_mha mha;
mha.mha_cmd = MHA_MAPSIZE_BSSBRK;
mha.mha_flags = 0;
mha.mha_pagesize = pagesize;
return
::memcntl
(
0,
0,
MC_HAT_ADVISE,
reinterpret_cast< caddr_t >( & mha ),
0,
0
)
== 0;
}
// Doesn't handle any exceptions.
void f()
{
// Chunk size should be a large-page size multiple
std::size_t const pagesize = largest_pagesize();
std::size_t const size = 512 * pagesize;
// Advise HAT to adopt a large-page size
// May trigger fix-up viewable with pmap -xs
// So from now on the heap will use large-page sizes
if ( !hat_advise_bssbrk( pagesize ) )
::perror( NULL );
// Friendly reserve a large-page multiple (chunk) from heap
// May trigger additional fix-up viewable with pmap -xs
// Reserved address range will also be viewable
void * p = ::memalign( pagesize, size );
// Optional, but useful to prevent swapping
if ( ::mlock( p, size ) != 0 )
::perror( NULL );
// Touch the reserved memory
// Triggers the actual allocation
::memset( p, '*', size );
if ( ::munlock( p, size ) != 0 )
::perror( NULL );
// Done
::free( p );
}
ALTERNATIVE 3
The previous alternative may be too invasive or extensive as the whole program's heap is reconfigured for a large-page size. With the previous approach I can't leave just the allocated (special) chunk backed by large pages. This is probably not what's desired for other ordinary casual allocations. Furthermore, trying to revert the heap pages back to the defaults, negatively affects previous (special) large-page allocations.Thanks again to Solaris the solution is easy.
Consider the following slight variations from the previous code:
inline bool hat_advise_va
(
void const * const p,
std::size_t const size,
std::size_t const pagesize
)
{
::memcntl_mha mha;
mha.mha_cmd = MHA_MAPSIZE_VA;
mha.mha_flags = 0;
mha.mha_pagesize = pagesize;
return
::memcntl
(
static_cast< caddr_t >
( const_cast< void * >( p ) ),
size,
MC_HAT_ADVISE,
reinterpret_cast< caddr_t >( & mha ),
0,
0
)
== 0;
}
// A typical usage pattern.
// Doesn't handle any exceptions.
void f()
{
// Chunk size should be a large-page size multiple
std::size_t const pagesize = largest_pagesize();
std::size_t const size = 512 * pagesize;
// Friendly reserve a large-page multiple (chunk) from heap
// May trigger additional fix-up viewable with pmap -xs
// Reserved address range will also be viewable
void * p = ::memalign( pagesize, size );
// Advise HAT to adopt a large-page size for the chunk
// May trigger fix-up viewable with pmap -xs
if ( !hat_advise_va( p, size, pagesize ) )
::perror( NULL );
// Optional, but useful to prevent swapping
if ( ::mlock( p, size ) != 0 )
::perror( NULL );
// Touch the reserved memory
// Triggers the actual allocation
::memset( p, '*', size );
// May immediately page-out parts of chunk
if ( ::munlock( p, size ) != 0 )
::perror( NULL );
// Done
::free( p );
}
By the way, if I want to check the page size backing a certain region:
inline std::size_t pagesize( void const * const p )
{
uint_t const request = MEMINFO_VPAGESIZE;
uint64_t output;
uint_t validity;
if ( ::meminfo
(
reinterpret_cast< uint64_t const * >( & p ),
1,
& request,
1,
& output,
& validity
)
== 0
)
// Is p a valid virtual address?
if ( validity & 1 )
// Has the virtual address been touched?
// Is there any memory page backing it?
if ( validity & 2 )
return output;
// No page is backing the virtual address
return 0;
}
For exemplifying the behavior of the code compiled (-g -m64) on an Intel x64, from a debugging session, let's take excerpts of the heap life-cycle from a series of:
$ pmap -xs `pgrep <program>` | head -25
The program's heap starts as follows:
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
0000000000400000 16 16 - - 4K r-x-- ...
0000000000413000 4 4 4 - 4K rw--- ...
0000000000414000 36 36 36 - 4K rw--- [ heap ]
000000000041D000 4 - - - - rw--- [ heap ]
000000000041E000 8 8 8 - 4K rw--- [ heap ]
0000000000420000 28 - - - - rw--- [ heap ]
0000000000427000 4 4 4 - 4K rw--- [ heap ]
0000000000428000 28 - - - - rw--- [ heap ]
000000000042F000 4 4 4 - 4K rw--- [ heap ]
0000000000430000 28 - - - - rw--- [ heap ]
0000000000437000 4 4 4 - 4K rw--- [ heap ]
FFFF80FFB8EB0000 4 4 - - 4K r-x-- ...
...
After the call to ::memalign():
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
...
0000000000437000 4 4 4 - 4K rw--- [ heap ]0000000000438000 1820 - - - - rw--- [ heap ]
00000000005FF000 4 4 4 - 4K rw--- [ heap ]
0000000000600000 1048576 - - - - rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
0000000040601000 216 - - - - rw--- [ heap ]
0000000040637000 4 4 4 - 4K rw--- [ heap ]
FFFF80FFB8EB0000 4 4 - - 4K r-x-- ...
...
After the call to ::mlock():
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
...
00000000005FF000 4 4 4 - 4K rw--- [ heap ]0000000000600000 1048576 1048576 - 1048576 - rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
After the call to ::memset():
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
...
00000000005FF000 4 4 4 - 4K rw--- [ heap ]0000000000600000 1048576 1048576 1048576 1048576 2M rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
After the call to ::munlock():
Address Kbytes RSS Anon Locked Pgsz Mode Mapped File
...
00000000005FF000 4 4 4 - 4K rw--- [ heap ]
0000000000600000 563200 563200 563200 - 2M rw--- [ heap ]00000000005FF000 4 4 4 - 4K rw--- [ heap ]
0000000022C00000 8192 8192 - - - rw--- [ heap ]
0000000023400000 477184 477184 477184 - 2M rw--- [ heap ]
0000000040600000 4 4 4 - 4K rw--- [ heap ]
...
Among other things, note that the next available heap address lies on a default page size, in the case of this system, a 4K page, instead of the largest page size, again, in the case of this system, a 2M page. This is exactly the fine-grained manual and dynamic control I was looking for, thanks to Solaris, of course!
Tuesday, June 18, 2013
DISM run sample 3.0
This is a sample run for DISM of the shared memory code sample 3.0:
In an attempt to better demonstrate the presumable advantage of DISM, please, consider the following changes in the original sample source code 3.0:
037: std::size_t const size =
12UL * 1024UL * 1024UL * 1024UL;
051: void * p = ::shmat( id, 0, SHM_PAGEABLE );
063: if ( ::mlock( p, size / 4 ) == 0 )
069: ::memset( p, '*', size / 2 );
075: switch ( ::munlock( p, size / 4 ) )
$ getent user_attr
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 178M reserved = 676M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.32 1.37% 8355M 34.0% 10.0G 35.9% 396K 0.01%
[system] 0.32 1.36% 8274M 33.6% 9.9G 35.5% - -
... 0.00 0.00% 81.0M 0.33% 116M 0.40% 228 0.00%
Getting shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 12G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 12884901888 3832 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.11% 8378M 34.1% 22.0G 78.8% 711K 0.02%
[system] 0.26 1.10% 8296M 33.7% 9.9G 35.5% - -
... 0.00 0.00% 82.5M 0.33% 12.1G 43.2% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2176K ...
...
Attaching to shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 492M allocated + 12G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.10% 8391M 34.1% 22.0G 78.8% 534 0.00%
[system] 0.26 1.09% 8308M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 82.6M 0.33% 12.1G 43.3% 432 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 2176K ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 - ... - rwxs- [ dism ... ]
...
Locking shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 3.5G allocated + 9.2G reserved = 13G used, 2.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.29 1.23% 11.1G 46.6% 25.0G 89.5% 704K 0.00%
[system] 0.29 1.22% 8308M 33.8% 12.9G 46.2% - -
... 0.00 0.00% 3154M 12.8% 12.1G 43.3% 1347 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 3074M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 3145728 ... 3145728 rwxs- [ dism ... ]
...
Using shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 2.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.27 1.13% 14.1G 59.1% 25.0G 89.5% 228 0.00%
[system] 0.27 1.12% 8309M 33.8% 12.9G 46.2% - -
... 0.00 0.00% 6225M 25.3% 12.1G 43.3% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 6146M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 6291456 ... 3145728 rwxs- [ dism ... ]
...
Unlocking shared memory.
Press<ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.30 1.27% 14.1G 59.1% 22.0G 78.8% 181K 0.00%
[system] 0.30 1.27% 8309M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 6225M 25.3% 12.1G 43.2% 432 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 6146M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 6291456 ... - rwxs- [ dism ... ]
...
Detaching shared memory.
Press<ENTER>
Success!
vlab-3 $ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 0 12884901888 3832 ... 0 group.staff
vlab-3 $ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.29 1.22% 14.1G 59.1% 22.0G 78.8% 228 0.00%
[system] 0.29 1.22% 14.1G 58.8% 9.9G 35.5% - -
... 0.00 0.00% 81.5M 0.33% 12.1G 43.2% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2180K ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FFB8F00000 1620 1024 ... - r-x-- libCstd.so.1
...
Removing shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 171M reserved = 672M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.11% 8385M 34.1% 10.0G 35.9% 228 0.00%
[system] 0.26 1.11% 8303M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 81.5M 0.33% 123M 0.43% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2180K ...
...
Exiting program.
Press<ENTER>
When Oracle Database fellows argues that "DISM can be dynamically resized depending on the application demand" they induce sysadmins to confusion or error.
As seen above throughout ipcs -mA on all the "output groups" the shared segment size never changes. What may change is the amount of memory the application chooses to lock, contrasting to ISM where the kernel always keep the whole segment locked.
Furthermore, they also believe that "Solaris does not allocate or reserve memory of size SGA_MAX_SIZE at instance startup". Again, for me, this is a misconception of them. I'm convinced that Oracle Database do reserve one or more large DISM segments, but that may not be apparent for a DBA, but the above output shows that Solaris does.
The behavior of ::shmget() is the same for ISM and DISM. The differences start, depending on how the segment is attached to the process. But before that no system resources are actually used, just accounted in the kernel (as a "promise" to the process which called ::shmget()).
Attaching DISM to a process, also just accounts pageable (as seen by pmap -x Mode flag) virtual memory to the process, but not yet any actual resources.
Locking a portion of (or the whole) DISM segment causes adjustments to virtual memory figures (allocated x reserved) because actual resources are allocated. Memory overhead is credited into kernel figures and the process resident size (RSS) also grows.
On the 5th "output group" we see that touching more memory than is locked (as in this sample code), causes even more pages to be allocated. In the example, we touch twice more than is currently locked, just to be more eye catching.
Unlocking DISM, reliefs the accounting of virtual memory but not necessarily frees physical memory (no pageouts) as there may not be memory pressure (as in this example).
And as with ISM, detaching shared memory, just unmaps it from the process, but not from the kernel. To free kernel resources the ::shmctl() must be issued.
In an attempt to better demonstrate the presumable advantage of DISM, please, consider the following changes in the original sample source code 3.0:
037: std::size_t const size =
12UL * 1024UL * 1024UL * 1024UL;
051: void * p = ::shmat( id, 0, SHM_PAGEABLE );
063: if ( ::mlock( p, size / 4 ) == 0 )
069: ::memset( p, '*', size / 2 );
075: switch ( ::munlock( p, size / 4 ) )
$ getent user_attr
...::::defaultpriv=basic,proc_lock_memory;...
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 178M reserved = 676M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.32 1.37% 8355M 34.0% 10.0G 35.9% 396K 0.01%
[system] 0.32 1.36% 8274M 33.6% 9.9G 35.5% - -
... 0.00 0.00% 81.0M 0.33% 116M 0.40% 228 0.00%
Getting shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 12G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 12884901888 3832 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.11% 8378M 34.1% 22.0G 78.8% 711K 0.02%
[system] 0.26 1.10% 8296M 33.7% 9.9G 35.5% - -
... 0.00 0.00% 82.5M 0.33% 12.1G 43.2% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2176K ...
...
Attaching to shared memory.
Press
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 492M allocated + 12G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.10% 8391M 34.1% 22.0G 78.8% 534 0.00%
[system] 0.26 1.09% 8308M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 82.6M 0.33% 12.1G 43.3% 432 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 2176K ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 - ... - rwxs- [ dism ... ]
...
Locking shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 3.5G allocated + 9.2G reserved = 13G used, 2.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.29 1.23% 11.1G 46.6% 25.0G 89.5% 704K 0.00%
[system] 0.29 1.22% 8308M 33.8% 12.9G 46.2% - -
... 0.00 0.00% 3154M 12.8% 12.1G 43.3% 1347 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 3074M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 3145728 ... 3145728 rwxs- [ dism ... ]
...
Using shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 2.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.27 1.13% 14.1G 59.1% 25.0G 89.5% 228 0.00%
[system] 0.27 1.12% 8309M 33.8% 12.9G 46.2% - -
... 0.00 0.00% 6225M 25.3% 12.1G 43.3% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 6146M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 6291456 ... 3145728 rwxs- [ dism ... ]
...
Unlocking shared memory.
Press
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 1 12884901888 3832 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.30 1.27% 14.1G 59.1% 22.0G 78.8% 181K 0.00%
[system] 0.30 1.27% 8309M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 6225M 25.3% 12.1G 43.2% 432 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 12G 6146M ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FC80000000 12582912 6291456 ... - rwxs- [ dism ... ]
...
Detaching shared memory.
Press
Success!
vlab-3 $ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 6.5G allocated + 6.2G reserved = 13G used, 5.9G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 0 12884901888 3832 ... 0 group.staff
vlab-3 $ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.29 1.22% 14.1G 59.1% 22.0G 78.8% 228 0.00%
[system] 0.29 1.22% 14.1G 58.8% 9.9G 35.5% - -
... 0.00 0.00% 81.5M 0.33% 12.1G 43.2% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2180K ...
...
$ pmap -x 3832 | head
3832: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
0000000000400000 8 8 ... - r-x-- shm_v04
0000000000411000 8 8 ... - rw--- shm_v04
0000000000413000 160 68 ... - rw--- [ heap ]
FFFF80FFB8F00000 1620 1024 ... - r-x-- libCstd.so.1
...
Removing shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 500M allocated + 171M reserved = 672M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.11% 8385M 34.1% 10.0G 35.9% 228 0.00%
[system] 0.26 1.11% 8303M 33.8% 9.9G 35.5% - -
... 0.00 0.00% 81.5M 0.33% 123M 0.43% 228 0.00%
$ prstat -c -p 3832 1 1
Please wait...
PID USERNAME SIZE RSS ...
3832 ... 5140K 2180K ...
...
Exiting program.
Press
When Oracle Database fellows argues that "DISM can be dynamically resized depending on the application demand" they induce sysadmins to confusion or error.
As seen above throughout ipcs -mA on all the "output groups" the shared segment size never changes. What may change is the amount of memory the application chooses to lock, contrasting to ISM where the kernel always keep the whole segment locked.
Furthermore, they also believe that "Solaris does not allocate or reserve memory of size SGA_MAX_SIZE at instance startup". Again, for me, this is a misconception of them. I'm convinced that Oracle Database do reserve one or more large DISM segments, but that may not be apparent for a DBA, but the above output shows that Solaris does.
The behavior of ::shmget() is the same for ISM and DISM. The differences start, depending on how the segment is attached to the process. But before that no system resources are actually used, just accounted in the kernel (as a "promise" to the process which called ::shmget()).
Attaching DISM to a process, also just accounts pageable (as seen by pmap -x Mode flag) virtual memory to the process, but not yet any actual resources.
Locking a portion of (or the whole) DISM segment causes adjustments to virtual memory figures (allocated x reserved) because actual resources are allocated. Memory overhead is credited into kernel figures and the process resident size (RSS) also grows.
On the 5th "output group" we see that touching more memory than is locked (as in this sample code), causes even more pages to be allocated. In the example, we touch twice more than is currently locked, just to be more eye catching.
Unlocking DISM, reliefs the accounting of virtual memory but not necessarily frees physical memory (no pageouts) as there may not be memory pressure (as in this example).
And as with ISM, detaching shared memory, just unmaps it from the process, but not from the kernel. To free kernel resources the ::shmctl() must be issued.
ISM run sample 3.0
This is a sample run for ISM of the shared memory code sample 3.0:
$ getent user_attr prime ...::::defaultpriv=basic,proc_lock_memory;...
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 170M reserved = 668M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.09% 7719M 31.4% 10.0G 35.8% 540K 0.00%
[system] 0.26 1.09% 7632M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 86.9M 0.35% 117M 0.40% 1280 0.00%
Getting shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 1.2G reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 1073741824 3253 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.25 1.05% 7720M 31.4% 11.0G 39.4% 317 0.00%
[system] 0.25 1.05% 7633M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 86.9M 0.35% 1144M 3.99% 317 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K
...
Attaching to shared memory.
Press<ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 1.5G allocated + 182M reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 1073741824 3253 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.25 1.07% 8741M 35.5% 11.0G 39.4% 1347 0.00%
[system] 0.25 1.06% 7631M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 1109M 4.51% 1148M 4.00% 1347 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 1028M 1026M ...
...
$ pmap -x 3253 | head
3253: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
08050000 8 8 ... - r-x-- shm_v04
08061000 4 4 ... - rwx-- shm_v04
08062000 128 44 ... - rwx-- [ heap ]
80000000 1048576 1048576 ... 1048576 rwxsR [ ism shmid=... ]
...
Locking shared memory.
Press<ENTER>
Success!
not relevant for ISM
Using shared memory.
Press<ENTER>
Success!
not relevant for ISM
Unlocking shared memory.
Press <ENTER> to continue...
Success!
not relevant for ISM
Detaching shared memory.
Press <ENTER> to continue...
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 1.5G allocated + 182M reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 1073741824 3253 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.24 1.03% 8742M 35.5% 11.0G 39.4% 228 0.00%
[system] 0.24 1.03% 8656M 35.2% 9.9G 35.4% - -
... 0.00 0.00% 85.9M 0.34% 1143M 3.99% 228 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K ...
...
$ pmap -x 3253 | head
3253: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
08050000 8 8 ... - r-x-- shm_v04
08061000 4 4 ... - rwx-- shm_v04
08062000 128 44 ... - rwx-- [ heap ]
FE380000 24 12 ... - rwx-- [ anon ]
...
Removing shared memory.
Press<ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 174M reserved = 668M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.30 1.25% 7718M 31.4% 10.0G 35.8% 9852 0.00%
[system] 0.30 1.25% 7632M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 85.9M 0.34% 120M 0.41% 228 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K ...
...
Exiting program.
Press<ENTER>
When Oracle Database fellows argues that "::shmget() requires swap reservation" they induce sysadmins to confusion or error.
As seen above (on the 2nd "output group") what happens is simply virtual memory reservation within the kernel. Note there's no other swap maneuver or switch between disk and physical memory and so on. The process that does the call doesn't even exhibit different figures.
With ISM, physical memory actually gets allocated only when the reserved shared segment is mapped in the calling process via ::shmat(). This is clearly seen in the 3rd "output group" above.
We also see that ::shmdt() doesn't really release any allocated memory as seen in the 4th "output group". The shared memory segment is unmapped from the process but continues to exist within the kernel.
It's only when ::shmctl() is called that memory is actually freed (fifth "output group").
$ getent user_attr prime ...::::defaultpriv=basic,proc_lock_memory;...
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 170M reserved = 668M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.26 1.09% 7719M 31.4% 10.0G 35.8% 540K 0.00%
[system] 0.26 1.09% 7632M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 86.9M 0.35% 117M 0.40% 1280 0.00%
Getting shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 1.2G reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 1073741824 3253 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.25 1.05% 7720M 31.4% 11.0G 39.4% 317 0.00%
[system] 0.25 1.05% 7633M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 86.9M 0.35% 1144M 3.99% 317 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K
...
Attaching to shared memory.
Press
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 1.5G allocated + 182M reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 1 1073741824 3253 ... 1 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.25 1.07% 8741M 35.5% 11.0G 39.4% 1347 0.00%
[system] 0.25 1.06% 7631M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 1109M 4.51% 1148M 4.00% 1347 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 1028M 1026M ...
...
$ pmap -x 3253 | head
3253: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
08050000 8 8 ... - r-x-- shm_v04
08061000 4 4 ... - rwx-- shm_v04
08062000 128 44 ... - rwx-- [ heap ]
80000000 1048576 1048576 ... 1048576 rwxsR [ ism shmid=... ]
...
Locking shared memory.
Press
Success!
not relevant for ISM
Using shared memory.
Press
Success!
not relevant for ISM
Unlocking shared memory.
Press <ENTER>
Success!
not relevant for ISM
Detaching shared memory.
Press <ENTER>
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 1.5G allocated + 182M reserved = 1.7G used, 17G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
m ... 0x1 ... 0 1073741824 3253 ... 0 group.staff
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.24 1.03% 8742M 35.5% 11.0G 39.4% 228 0.00%
[system] 0.24 1.03% 8656M 35.2% 9.9G 35.4% - -
... 0.00 0.00% 85.9M 0.34% 1143M 3.99% 228 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K ...
...
$ pmap -x 3253 | head
3253: ./shm_v04
Address Kbytes RSS ... Locked Mode Mapped File
08050000 8 8 ... - r-x-- shm_v04
08061000 4 4 ... - rwx-- shm_v04
08062000 128 44 ... - rwx-- [ heap ]
FE380000 24 12 ... - rwx-- [ anon ]
...
Removing shared memory.
Press
Success!
$ swap -lh; swap -sh
swapfile dev swaplo blocks free
/dev/swap - 4K 4.0G 3.9G
total: 496M allocated + 174M reserved = 668M used, 18G available
$ ipcs -mA
IPC status ...
T ... KEY ... NATTCH SEGSZ CPID ... ISMATTCH PROJECT
Shared Memory:
$ zonestat 1 1
Collecting data for first interval...
Interval: 1, Duration: 0:00:01
SUMMARY Cpus/Online: 24/24 PhysMem: 23.9G VirtMem: 27.9G
---CPU---- --PhysMem-- --VirtMem-- --PhysNet--
ZONE USED %PART USED %USED USED %USED PBYTE %PUSE
[total] 0.30 1.25% 7718M 31.4% 10.0G 35.8% 9852 0.00%
[system] 0.30 1.25% 7632M 31.0% 9.9G 35.4% - -
... 0.00 0.00% 85.9M 0.34% 120M 0.41% 228 0.00%
$ prstat -c -p 3253 1 1
Please wait...
PID USERNAME SIZE RSS ...
3253 ... 3832K 1720K ...
...
Exiting program.
Press
When Oracle Database fellows argues that "::shmget() requires swap reservation" they induce sysadmins to confusion or error.
As seen above (on the 2nd "output group") what happens is simply virtual memory reservation within the kernel. Note there's no other swap maneuver or switch between disk and physical memory and so on. The process that does the call doesn't even exhibit different figures.
With ISM, physical memory actually gets allocated only when the reserved shared segment is mapped in the calling process via ::shmat(). This is clearly seen in the 3rd "output group" above.
We also see that ::shmdt() doesn't really release any allocated memory as seen in the 4th "output group". The shared memory segment is unmapped from the process but continues to exist within the kernel.
It's only when ::shmctl() is called that memory is actually freed (fifth "output group").
Subscribe to:
Posts (Atom)