(PRM) Script for better process control RAM, CPU

T

his tutorial works for CentOS only. For your specific OS tutorial use the tags or serach function in the sidebar area.

 

This software from R-fx Networks by Ryan MacDonald is a free, smart and powerfull bash script for monitoring and controling CPU, Memory, Processes & Run (Elapsed) Time in Linux & BSD environment. PRM has very flexible settings controlled via global variables config file or via user custom rule files. Administator can define own rules
by defining positive variables or using ignore options also configure soft/hard kill triggers, wait/recheck timings and to send kill signals to parent/children process trees.

You can use it for autoamtically kill apache processes eating too much RAM or prevent scripts executed under specified user from running for hours at a time.

PRM also includes SPRI (System Priority) script which allows set automatic priority for services you want to run smoothest http://www.rfxn.com/projects/system-priority/

 

Features:

- global resource limits
- per-process/per-user rule based resource limits
- rules only run mode
- alert only run mode
- ignore root processes option
- process list global ignore file
- user based global ignore file
- command based global ignore file
- regex based global and/or per-rule ignore variable
- global scoped resource limits
- per-process or per user rule based resource limits
- set custom kill signals (i.e: SIGHUP)
- parent process kill option to terminate an entire process tree
- kill trigger/wait times allow rechecking usage over a period of time
- kill restart option to execute a custom restart command
- all kill/resource/ignore options can be global or rule defined
- easy to configure percentage of total CPU & MEM limits
- total number of processes limits
- elapsed run time limits

The readme file is pretty self explanatory - be sure to read it after installation as I wouldn't double the all info in here.

1. Installation

 
wget http://www.rfxn.com/downloads/prm-current.tar.gz
 

Now extract archive and after this run the install script

$ ./install.sh


.: PRM installed
Install path: yourpath
Config path: yourpath/conf.prm
Executable path:
CronJob path: /etc/cron.d/prm"
 

As you can see:

$ whereis prm
prm: /usr/local/sbin/prm /usr/local/prm
 

2. Setup

I will describe a simple setup of PRM as the config file is pretty well documented and self-explanatory.

The config file is located at /usr/local/prm/conf.prm

We will edit the file now:

$ nano /usr/local/prm/conf.prm
 

This will be basic-test setup, ale other rules you have to write on your own depending your needs.
Make sure you have those options set, for a start.

# For now we do want alerts,
# later you might this turn off
EMAIL_ALERT=1
EMAIL_ADDR="username"

# For now, we want to use only rules that we write
# not the global settings
RULES_ONLY="1"

# For now we need only testing, if everything will be OK
# then turn it to "0"
ALERT_ONLY="1"

IGNORE_ROOT="1"
 

Save and exit.

As I said, PRM already includes SPRI, which is another clever script for services priority, so you might want to check the directory /prios now and tune priority settings according to your needs, but this isn' required as default settings will suit most needs.

So now we create our first rule file in /usr/local/prm/rules/

As the readme says, we can create two type of rule files:

USERNAME.user
COMMAND.cmd

For example, to create a rule for user "mike" you would create rules/mike.user, to create a rule for apache processes you would create rules/httpd.cmd or for the apache user "nobody" you would create rules/nobody.user. It is important to remember that .cmd rules only apply to the specific command process but user rules apply to ALL processes running under that user.


So, let's say we want to take care of our Apache:

We create a rule file:

$ touch /usr/local/prm/rules/httpd.cmd
$ nano /usr/local/prm/rules/httpd.cmd
 

And paste this into:

MAX_CPU="50"
MAX_MEM="13"
MAX_PROC="25"
# We don't use ETIME for now, so set this to 0
MAX_ETIME="0"
KILL_TRIG="3"
KILL_WAIT="10"
# We need to kill parent here otherwise the HTTP Request that spawned the
# script we are trying to kill, will probably just respawn it.
KILL_PARENT="1"
KILL_SIG="9"
KILL_RESTART_CMD="/etc/init.d/httpd restart"
 

Save rule and exit editor.

Now we have PRM configured, our first rule is set we need a dry run.
Remeber that the default execution of PRM is handled through /etc/cron.d/prm and set to run at 5 minute intervals, you may increase this at your own discretion. You are warned though, that if your system has hundreds of processes with many process events on a run of PRM, it can take a couple of minutes to complete.

So You do a dry run for to see how long it takes on Your server to complete and then adjust the cronjob interval at /etc/cron.d/prm

$ prm -c

Process Resource Monitor (PRM) v1.1.4
(C) 2002-2012, R-fx Networks
(C) 2012, Ryan MacDonald
This program may be freely redistributed under the terms of the GNU GPL v2

prm[20516]: checking proc pid:3731 {user:postfix cmd:tlsmgr}
prm[20516]: checked 44 processes in 12 seconds
 

That's it. For now you can watch for any email alerts if the PRM finds out a state when your rule matches.
If you think it works in your environment then you may ALERT_ONLY="0".