pScheduler offers the ability to control what tasks may be run, by whom and with what parameters. This is achieved by writing and installing a limit configuration file.
A number of commented samples may be found in /usr/share/doc/pscheduler/limit-examples on systems with the pScheduler bundle installed. The limit configuration that ships with the perfSONAR toolkit may be found in the toolkit sources.
The file is JavaScript Object Notation (JSON) containing a single object with the four pairs shown here:
{
"#": "Skeletal pScheduler limit configuration",
"identifiers": [ ... ],
"classifications": [ ... ],
"limits": [ ... ],
"applications": [ ... ]
}
Each pair is described in the sections below.
All JSON read by pScheduler supports in-line commenting by ignoring any pair whose key begins with a pound sign (#):
{
"#": "This is a comment.",
"#This": "is also a comment.",
"This": "#is not a comment.",
}
Note that this behavior is not part of ECMA 404, the JSON standard.
The first phase of vetting a task or run is identification, where attributes of the arriving request are used to create a list of narrow categories into which the requester fits.
The identifiers section of the limit configuration contains an array of identifier objects, each containing the following pairs:
For example:
{
"identifiers": [
{
"name": "partners-bio",
"description": "Research Partners in biology",
"type": "ip-cidr-list",
"data": {
"cidrs": [ "192.0.2.0/24", "198.51.100.0/24" ]
},
"invert": false
},
{
"name": "local",
"description": "Requesters on the local system",
"type": "localif",
"data": { }
},
{
"name": "everyone",
"description": "All requesters",
"type": "always",
"data": { }
},
],
...
}
The always identifier unconditionally identifies every requester, useful in catch-alls.
Its data is an empty object:
{
"data": { }
}
There are exactly two useful configurations of this identifier:
{
"name": "everybody",
"description": "An identifier that identifies every requester",
"type": "always",
"data": { }
}
{
"name": "nobody",
"description": "An identifier that identifies no requesters",
"type": "always",
"data": { },
"invert": true
}
The hint identifier matches information about the requester to make identifications.
Its data is an object containing the following pairs:
requester, a string containing the IP address of the host making the request, and server, a string containing the IP address of the interface on the local system where the request arrived.
match - A StringMatch object. (See Standard Objects.)
For example:
{
"name": "internal",
"description": "Requests arriving on our internal-facing interface",
"data": {
"hint": "server",
"match": {
"style": "exact",
"match": "198.51.100.23"
}
}
}
The ip-cidr-list identifier determines whether or not the IP address of the host making a request falls into any of a list of Classless Inter-Domain Routing <https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing>`_ (CIDR) blocks.
Its data is an object containing the following pairs:
For example:
{
"name": "partners",
"description": "Networks used by research partners",
"type": "ip-cidr-list",
"data": {
"cidrs": [
"203.0.113.62",
"192.168.19.0/24",
"192.168.84.0/24",
"2001:db8::1234",
"fc00:1bad:cafe::/48",
"fc00:dead:beef::/48"
]
}
}
The ip-cidr-list-url identifier serves the same purpose as ip-cidr-list but downloads the list of CIDRs from a URL and periodically updates it.
Its data is an object containing the following pairs:
Note that this identifier will continue to use the list it last successfully downloaded until an update can be successfully retrieved.
For example, this identifier downloads ESNet’s list of CIDRs for research and education networks, updates it daily with four-hour retries on failure and excludes the private networks defined by RFC 1918:
{
"name": "r-and-e",
"description": "Requests from research and education networks",
"type": "ip-cidr-list-url",
"data": {
"source": "http://stats.es.net/sample_configs/pscheduler/ren",
"update": "P1D",
"retry": "PT4H",
"exclude": [
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16"
],
"fail-state": false
}
}
The ip-cymru-bogon identifier determines whether or not the requester’s address is in Team Cymru’s Bogon Refernce List.
Its data is an object containing the following pairs:
Note that this identifier uses the Domain Name Service to check whether or not an address is in the list, and therefore its use requires that the host be able to resolve hosts on the public Internet. This system works with caching DNS servers, so direct access to the internet is not required.
For example, this identifier checks incoming request addresses, excludes three of the RFC1918 blocks, gives up after one second and does not identify the requester as a bogon if a definitive answer cannot be found:
{
"name": "bogons",
"description": "Requests arriving from bogon/martian addresses",
"type": "ip-cymru-bogon",
"data": {
"exclude": [
"10.10.0.0/16",
"192.168.86.0/24",
"192.168.99.0/24"
],
"timeout": "PT1S",
"fail-result": false
}
}
The ip-reverse-dns identifier attempts to reverse-resolve the requester’s IP address to a fully-qualified domain name and matches it against a pattern.
Its data is an object containing the following pairs:
As a security measure, the fully-qualified domain name found during reverse resolution will be forward-resolved to an IP which must match that of the requester.
For example, this identifier determines whether or not the incoming requester’s fully-qualified domain name falls within example.org, giving up after two seconds:
{
"name": "example-dot-org",
"description": "Requests arriving from example.org IPs",
"type": "ip-reverse-dns",
"data": {
"match": {
"style": "regex",
"match": "\\.example\\.org$"
},
"timeout": "PT2S"
}
}
The localif identifier determines whether or not the requester’s IP address is bound to an interface on the local system.
Its data is an empty object:
{
"data": { }
}
For example:
{
"name": "local-requester",
"description": "Requests arriving from local interfaces",
"type": "localif",
"data": { }
}
Once a list of identifiers is determined, the second phase is grouping them into broader categories called classifiers. Classifiers are simple groups containing a list of one or more identifiers.
The classifiers section of the limit configuration contains an array of classifier objects, each containing the following pairs:
For example:
{
...
"classifiers": [
{
"name": "friendlies",
"description": "Requesters we like",
"identifiers": [ "local", "partners", "r-and-e" ]
},
{
"name": "hostiles",
"description": "Requesters we don't want using the system",
"identifiers": [ "bogons", "example-dot-org" ]
},
{
"name": "neutrals",
"description": "Requesters we neither like nor dislike",
"identifiers": [ "everybody" ]
},
...
}
Note that the neutrals classification will include all requesters, which makes it overlap with friendlies and hostiles. As will be illustrated later, the narrower classifications can be used to allow or deny tasks before the wider ones.
The third phase of vetting a task is determining whether or not its parameters fall within acceptable values. Each limit is evaluated and either passes (i.e., the task parameters fell within the limit’s restrictions) or fails (i.e., it did not).
The limits section of the limit configuration is nearly identical to the identifiers section and contains the following pairs:
For example:
{
...
"limits": [
{
"name": "always",
"description": "Always passes",
"type": "pass-fail",
"data": {
"pass": true
}
},
{
"name": "innocuous-tests",
"description": "Tests that are harmless",
"type": "test-type",
"data": {
"types": [ "idle", "latency", "rtt", "trace" ]
}
},
{
"name": "throughput-default-template",
"description": "Template for throughput defaults",
"type": "test",
"data": {
"test": "throughput",
"limit": {
"duration": {
"range": { "lower": "PT5S", "upper": "PT60S" }
}
}
},
{
"name": "throughput-default-udp",
"description": "UDP throughput for all requesters",
"clone": "throughput-default-template",
"data": {
"limit": {
"bandwidth": {
"range": { "lower": "1", "upper": "800K" },
}
"udp": { "match": true }
}
}
},
{
"name": "throughput-default-tcp",
"description": "TCP throughput for all requesters",
"clone": "throughput-default-template",
"data": {
"limit": {
"bandwidth": {
"range": { "lower": "1", "upper": "50M" },
}
"udp": { "match": false }
}
}
}
],
...
}
The pass-fail limit will either pass or fail depending on a value in its data.
Its data is an object containing the following pair:
For example:
{
"name": "never",
"description": "Fail to pass",
"type": "pass-fail",
"data": {
"pass": false
}
}
The run-daterange limit tests to see whether the time range for a run falls within a specified range.
Its data is an object containing the following pairs:
Note that limits of this type are not evaluated and will be considered to have passed when determining whether a task will be allowed on the system.
For example:
{
"name": "summer-2017",
"description": "The summer of 2017",
"type": "run-daterange",
"data": {
"start": "2017-06-21T00:00:00",
"end": "2017-09-22T23:59:59"
}
}
The run-daterange limit tests to see whether attributes the time range for a run matches those specified.
Its data is an object containing the following pairs. The format of the pairs is described below.
All pairs are optional.
Each pair consists of a key (e.g., month) and an array of individual numbers or ranges. Each range is an object containing the following pairs:
Note that this limits of this type are not evaluated and will be considered to have passed when determining whether a task will be allowed on the system.
For example:
{
"name": "not-in-maint-window",
"description": "Outside weekly maintenance windows (Wed & Sun, 2 and 4-8 a.m.)",
"type": "run-schedule",
"data": {
"weekday": [ 3, 7 ],
"hour": [ 2, { "lower": 4, "upper": 7 } ],
"overlap": true
"invert": true
}
}
The test limit compares the parameters of a proposed test against a template containing acceptable values.
Its data is an object containing the following pairs:
For example:
{
"name": "throughput-udp",
"description": "Limits for UDP throughput tests",
"type": "test",
"data": {
"test": "throughput",
"limit": {
"duration": { "range": { "lower": "PT5S", "upper": "PT60S" } },
"bandwidth": { "range": { "lower": "1", "upper": "50M" } },
"udp": { "match": true }
}
}
The test-type limit compares the type of the proposed test to a list of test types.
Its data is an object containing the following pair:
For example:
{
"name": "inoccuous-tests",
"description": "Tests that are harmless",
"type": "test-type",
"data": {
"types": [ "idle", "latency", "rtt", "trace" ]
}
}
The final phase of vetting a task or run is determining whether or not its parameters make it permissible. This is accomplished by evaluating a series of limit applications, each of which ties a classifier to a series of conditions which must be met before approval can happen.
Each limit application is a JSON object consisting of the following:
The system will evaluate each application in sequence. (This process is described in detail in Applying Limit Requirements, below.) If an application passes (i.e., its conditions will allow the task or run to happen), the task or run is permitted. If it fails and stop-on-failure is true, it is denied. If if fails and stop-on-failure is false, the next application in the list is evaluated. If the end of the list is reached with no application having passed, the task or run is denied.
For example:
{
...
"applications": [
{
"description": "Allow users on the local system to do anything",
"classifier": "local-requester",
"apply": [
{
"require": "all",
"limits": [ "always" ]
}
]
},
{
"description": "What we allow guests to do",
"classifier": "guests",
"apply": [
{
"require": "any",
"limits": [
"innocuous-tests",
"guest-throughput",
"guest-rtt"
]
}
],
"stop-on-failure": true
}
]
}
The first application allows any requester in the local-requester classification to run anything because it applies the always limit, which always passes. The second application alows requesters in the guests classifier be runing any of the harmless tests or a throughput or round-trip time test that meets predefined limits for guests. Failing both of those will result in denial because the policy is to deny unless explicitly allowed.
Each limit requirement is a JSON object containing the following:
- none - Consider the requirement met if none of the limits passes.
- one - Consider the requirement met if exactly one of the limits passes.
- any - Consider the requirement met if at least one of the limits passes.
- all - Consider the requirement met only if all of the limits pass.
pScheduler includes a validate-limits command which can be used to verify that a limit configuration is valid during development and prior to installation on the system.
To validate limits in a file:
% pscheduler validate-limits valid-limits.conf
Limit configuration is valid.
% pscheduler validate-limits invalid-limits.conf
Invalid limit file: At /: Additional properties are not allowed (u'notvalid' were unexpected)
To validate the installed configuration, become root and execute:
# pscheduler validate-limits
Limit configuration is valid.
The command will exit with a status of 0 if the limit file was valid or nonzero if it was not. Errors will be sent to the standard error and a message indicating that the configuration is valid will be sent to the standard output if it is a TTY or the --quiet switch is not in effect.
Details on command-line switches and sample invocations can be obtained by running the command pscheduler validate-limits --help.
The limit configuration is installed in /etc/pscheduler/limits.conf and must be readable by the pscheduler user. The recommended file attributes are owner root, group pscheduler and permissions 0644.
pScheduler server automatically detect changes to the limit configuration and put them into effect upon the arrival of the first request that requires checking limits or 15 seconds, whichever is longer. Changes to the limit file are noted in the pScheduler log (usually /var/log/pscheduler/pscheduler.log), as are notifications of problems.
If the configuration file does not exist, is removed or fails to load, pScheduler will enforce no limits and grant every task request it receives. For this reason, it is strongly recommended that configurations be verified as described above before they are installed.
This section describes standard JSON objects used in the limit configuration.
Content in this section is forthcoming.
StringMatch is a JSON object containing the following pairs:
- exact - The compared string must be exactly equal to match.
- contains - The match string must be contained somewhere within the compared string.
- regex - The compared string must match the Python 2 regular expression specified in match.
For example, this StringMatch looks for an empty string or one containing a vowel:
{
"style": "regex",
"match": "(^$|[aeiou])"
}
This section describes standard types of objects used by the test limit.
For example:
{
"match": false
}
For example:
{
"range": { "lower": 5, "upper": 8 }
}
For example:
{
"match": [ 2, 4, 6, 8 ]
}
For example:
{
"range": { "lower": 0, "upper": 19 }
}
For example:
{
"match": [ 0, 2, 4, 6, 8 ]
}
For example:
{
"range": { "lower": "PT15S", "upper": "PT1M" }
}
For example:
{
"range": { "lower": "600K", "upper": "5G" }
}
For example:
{
"match": 6
}
For example:
{
"enumeration": [ 4, 6 ]
}
For example:
{
"range": { "lower": 0.25, "upper": 1.0 }
}
For example:
{
"match": {
style": "regex",
"match": "platypus",
"invert": true
}
}
Note that it is possible to have invert in both the limit and the match StringMatch object.