WLM uses the concept of resource sets (or rsets) to restrict the processes in a given class to a subset of the system's physical resources. In WLM, the physical resources managed are the memory and the processors. A valid resource set is composed of memory and at least one processor.
Using SMIT or Web-based System Manager, a system administrator can define and name resource sets containing a subset of the resources available on the system. Then, using the WLM administration interfaces, root user or a designated superclass administrator can use the name of the resource set as the rset attribute of a WLM class. From then on, every process assigned to this WLM class is dispatched only on one of the processors in the resource set, effectively separating workloads for the CPU resource.
All of the current systems have only one memory domain shared by all the resource sets, so this method does not physically separate workloads in memory.
The rset registry services enable system administrators to define and name resource sets so that they can then be used by other users or applications. To alleviate the risks of name collisions, the registry supports a two-level naming scheme. The name of a resource set is in the form name_space/rset_name. Both the namespace and rset_name can each be 255 characters in length, are case-sensitive, and may contain only uppercase and lowercase letters, numbers, underscores, and periods (.). The namespace of sys is reserved by the operating system and used for rset definitions that represent the resources of the system.
The rset definition names are unique within the registry name space. Adding a new rset definition to the registry using the same name as an existing rset definition causes the existing definition to be replaced with the new definition, given the proper permission and privilege. Only root can create, modify, and delete resource sets and update the in-core rset data base, using SMIT or Web-based System Manager.
Each rset definition has an owner (user ID), group (group ID), and access permissions associated with it. These are specified at the time the rset definition is created and exist for the purpose of access control. As is the case for files, separate access permissions exist for the owner, group and others that define whether read and/or write permission has been granted. Read permission allows an rset definition to be retrieved while write permission allows an rset definition to be modified or removed.
System Administrator defined rset definitions are kept in the /etc/rsets stanza file. The format of this file is not described, and users must manipulate rsets through the SMIT or Web-based System Manager interfaces to prevent future potential compatibility problems if the file format is modified. As is the case for WLM class definitions, the rset definitions must be loaded into kernel data structures before they can be used by WLM.
Rather than giving an extensive definition of resource sets and how to create them from basic building blocks called system RADs (Resource Access Domains), the following example details what is in a resource set and how to create new one.
The following example is on a 24-way system. The system administrator wants to create a resource set containing processors 0 to 5, and use it in WLM configuration to restrict all processes of a superclass to these six processors. This example uses SMIT, but the same steps can be done with Web-based System Manager.
The first step is to create and name the resource set. The system administration gets to the Resource Set Management menu either from the initial menu by selecting Performance & Resource Scheduling and then Resource Set Management or by using the fast path smit rset.
The Manage Resource Set Database option is used to create, modify, or delete rset. Before selecting Manage Resource Set Database, select List All System RADs to see what building blocks are available from which to create the resource sets:
COMMAND STATUS Command: OK stdout: yes stderr: no Before command completion, additional instructions may appear below. T Name Owner Group Mode CPU Memory Resources r sys/sys0 root system r----- 24 98298 sys/sys0 r sys/node.00000 root system r----- 24 98298 sys/sys0 r sys/mem.00000 root system r----- 0 98298 sys/mem.00000 r sys/cpu.00023 root system r----- 1 0 sys/cpu.00023 r sys/cpu.00022 root system r----- 1 0 sys/cpu.00022 r sys/cpu.00021 root system r----- 1 0 sys/cpu.00021 r sys/cpu.00020 root system r----- 1 0 sys/cpu.00020 r sys/cpu.00019 root system r----- 1 0 sys/cpu.00019 r sys/cpu.00018 root system r----- 1 0 sys/cpu.00018 r sys/cpu.00017 root system r----- 1 0 sys/cpu.00017 r sys/cpu.00016 root system r----- 1 0 sys/cpu.00016 r sys/cpu.00015 root system r----- 1 0 sys/cpu.00015 r sys/cpu.00014 root system r----- 1 0 sys/cpu.00014 r sys/cpu.00013 root system r----- 1 0 sys/cpu.00013 r sys/cpu.00012 root system r----- 1 0 sys/cpu.00012 r sys/cpu.00011 root system r----- 1 0 sys/cpu.00011 r sys/cpu.00010 root system r----- 1 0 sys/cpu.00010 r sys/cpu.00009 root system r----- 1 0 sys/cpu.00009 r sys/cpu.00008 root system r----- 1 0 sys/cpu.00008 r sys/cpu.00007 root system r----- 1 0 sys/cpu.00007 r sys/cpu.00006 root system r----- 1 0 sys/cpu.00006 r sys/cpu.00005 root system r----- 1 0 sys/cpu.00005 r sys/cpu.00004 root system r----- 1 0 sys/cpu.00004 r sys/cpu.00003 root system r----- 1 0 sys/cpu.00003 r sys/cpu.00002 root system r----- 1 0 sys/cpu.00002 r sys/cpu.00001 root system r----- 1 0 sys/cpu.00001 r sys/cpu.00000 root system r----- 1 0 sys/cpu.00000
As mentioned earlier, sys/sys0 represents the whole system (in this case, a 24-way SMP with 96GB of memory). This set is what processes in those WLM classes that do not specify a rset attribute potentially have access to. Later, we will select the memory and some of the CPUs when creating our resource set.
Select Manage Resource Set Database, then select Add a Resource Set to the Database. From the resulting screen, enter or select the Name Space and Resource Set Name and then select the other attributes of the resource sets. In our case, we are creating a new namespace and resource set, so we enter the character strings for the Name Space and Resource Set Name fields.
Fill in the other fields by selecting from lists. For the Resources field, select from a list of the System RADs. In this case, select the lines corresponding to the memory and CPUs 0 to 5 (sys/cpu.00000 to sys.cpu.00005). Remember that only processor sets are supported. Press Enter after the selections to create a new rset named admin/proc0_5.
At this point, a new rset is created in /etc/rsets. To use the new rset, add it into the kernel data structures by selecting Reload Resource Set Database. This menu gives you the option to reload the data base now, at next boot or both. The first time you create a new resource set, select both so that your administrator-defined resource sets are loaded after each reboot and can be used by WLM. After that, you would probably select now.
Now that we have our new resource set, we will use it in WLM using the new_config defined in the previous article.
We want to go back to the definition of the superclass super1 and set up the rset attribute to use the new rset. Using SMIT, go to the Workload Manager's main menu (fast path smit wlm), then to Work on alternate configurations and on to Select a configuration and select new_config. We then go back to the main menu and from there to Change / Show Characteristics of a class and on to General characteristics of a class. Enter or select super1 as the class name and press Enter to display the class attributes.
Scroll to the rset field, press F4 to get the list of available rsets, and select the newly created admin/proc0_5. After you have selected the new rset and committed the modification, the classes file on disk is changed.
After starting (or updating if it was already running) WLM with the new class definition as explained in the previous article, the following is a quick test to show the effect of the resource set on super1:
root # wlmstat CLASS CPU MEM BIO Unclassified 0 0 0 Unmanaged 0 0 0 Default 8 0 0 Shared 0 0 0 System 0 0 0 super1 25 0 0 super2 0 0 0 super2.Default 0 0 0 super2.Shared 0 0 0 super2.sub1 0 0 0 super2.sub2 0 0 0 root #
This output shows that the 90 CPU bound processes, which otherwise unconstrained would take up 100% of the CPU (almost four times over if it were possible), use just 25% because they are limited to run on CPUs 0 to 5.
UID PID PPID C STIME TTY TIME CMD wlmu3 11234 1 95 15:11:45 0 3:28 loop ... ... ... ...
Go to the Show a Process Partition entry of the rset SMIT menu. Enter the PID of the process we are interested in (or select it from the list of processes displayed with F4 key) and press Enter. For example:
COMMAND STATUS Command: OK stdout: yes stderr: no Before command completion, additional instructions may appear below. CPU Memory Resources 6 98298 sys/mem.00000 sys/cpu.00005 sys/cpu.00004 sys/cpu.00003 sys/cpu.00002 sys/cpu.00001 sys/cpu.00000
Selecting a process from a class without a specified rset attribute (here the init process) shows the following default resource set:
CPU Memory Resources 24 98298 sys/sys0
Using rsets is an effective way to isolate workloads from one another as far as the CPU is concerned. By separating two different workloads into two classes and giving each class a different subset of the CPUs, you can make sure that the two workloads will never compete for CPU with one another. Of course, they still compete for physical memory and I/Os.