Overview
The perfSONAR MeshConfig software allows an administrator to generate a central file that defines measurements for a set of hosts that read the published file. The file is defined by the administrator in Apache config format. This document covers this format and the complete listing of options available.
Each configuration file consists of the following elements:
- A list of description providing a human-readable string that names your mesh. See description Directive for details on defining this element.
- A list of administrators responsible for the configuration file. See <administrator> Directive for details on defining this element.
- A list of organizations including the sites and hosts that they contain. See <organization> Directive for details on defining this element.
- A list of test specification that describe the parameters of the tests to be run. See <test_spec> Directive for details on defining this element.
- A list of groups the describe the topology of tests and the hosts they contain. See <group> Directive for details on defining this element.
- A list of tests that connect the previously defined test specifications and groups. See <test> Directive for details on defining this element.
- An optional set of host classes that allow for dynamic joining of tests to groups. See <host_class> Directive for details on defining this element.
Grouping
<organization> Directive
| Description: | An optional construct that groups together sites and hosts by the organization to which they belong |
|---|
| Syntax: | <organization>...</organization> |
|---|
| Contexts: | top level |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.3 and later |
|---|
The <organization> directive is strictly of purposes of grouping elements managed by the same entity together. In addition to keeping sites and hosts in a logical order, the description field can also be matched on when defining host classes.
<site> Directive
| Description: | An optional construct that groups an organization’s hosts together, generally based on location |
|---|
| Syntax: | <site>...</site> |
|---|
| Contexts: | organization |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.3 and later |
|---|
A site can group hosts within an organization together. For example, if an organization has multiple campuses with two hosts deployed at each, then the site directive can be used to group the hosts at each campus. This example is illustrate below:
<site>
description Site 1
<host>...</host>
<host>...</host>
</site>
<site>
description Site 2
<host>...</host>
<host>...</host>
</site>
The <site> may have additional significance for tools that display data. For example, a MaDDash configuration can plot data on the same graph for hosts belonging to the same site. This is especially useful when different types of measurement data are spread across multiple hosts, such as throughput and one-way delay data.
Hosts
<host> Directive
| Description: | Describes a host that will be used as an endpoint in measurements |
|---|
| Syntax: | <host>...</host> |
|---|
| Contexts: | top level, organization, site |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.3 and later |
|---|
The host object is a critical part of your mesh configuration. You may define a host object at the top level of the file or within an organization or site. The latter two options simply provide more metadata about the host that can be used for documentation purposes or for hos class matches. Inside the construct you will define all the addresses which may be used for performing network measurements to/from this host as well as some additional metadata.
no_agent Directive
| Description: | A boolean indicating that his host does NOT read the mesh |
|---|
| Syntax: | no_agent 0|1 |
|---|
| Default: | 0 |
|---|
| Contexts: | host |
|---|
| Occurrences: | Zero or One |
|---|
| Compatibility: | 3.3 and later |
|---|
If set to 1, this value indicates that the host in question is not reading the mesh. This is significant because it means those that test to it are responsible for initiating tests in both directions and storing the results. The default is 0 meaning other testers can assume this host is reading the mesh and will initiate its portion of the tests accordingly.
<address> Directive
| Description: | An IPv4 address, IPv6 address, or hostname belonging to a host |
|---|
| Syntax: | address ADDRESS or <address>...</address> |
|---|
| Default: | 0 |
|---|
| Contexts: | host |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.3 or later for simple form, 3.5 and later for complex form |
|---|
An address is a required element of a host that defines an IPv4address , IPv6 address or hostname assigned to an interface on the host. You can and likely will define multiple of these for a single a host. If using a hostname it is not required to also define theIP addresses to which the hostname maps as the tools will do the look-ups automatically.
You can define address in one of two forms. In the simple form you simply provide the address. Example:
In the complex form you make it a block and can optionally add the tags field to label the address. Tags are used when defining a host class. Example:
<address>
address 10.0.1.1
tag latency
</address>
address Directive
| Description: | In the complex form of <address>, a simple string representation of the IPv4, IPv6 address or hostname |
|---|
| Syntax: | address ADDRESS |
|---|
| Contexts: | address |
|---|
| Occurrences: | Zero or One |
|---|
| Compatibility: | 3.3 and later |
|---|
Storing Results
<measurement_archive> Directive
| Description: | Defines where measurement results are stored |
|---|
| Syntax: | <measurement_archive>...</measurement_archive> |
|---|
| Contexts: | top level, organization, site, host |
|---|
| Occurrences: | One per <measurement_archive> type |
|---|
| Compatibility: | 3.3 or later |
|---|
The <measurement_archive> defines where measurements of a certain type are stored. Depending on where it is placed in the document it can have the following semantics:
- If placed in the <host> directive, then all measurements initiated by that host(and matching the measurement archive type) will be stored in the provided MA
- If placed in the <site> directive, all measurements initiated by hosts defined in the site will use the provided measurement archive UNLESS a <measurement_archive> is defined in the <host> directive.
- If placed in the <organization> directive, all measurements initiated by hosts defined in the organization will use the provided measurement archive UNLESS a <measurement_archive> is defined in the <site> directive OR the <host> directive (with <host> given preference).
- If placed in the top level, then all hosts will use the defined <measurement_archive> UNLESS there is <measurement_archive> defined in the <host>, <site>, or <organization> (with preference given in that order).
type Directive
| Description: | The type of measurement archive |
|---|
| Syntax: | type perfsonarbuoy/bwctl|perfsonarbuoy/owamp|pinger|traceroute |
|---|
| Contexts: | measurement archive |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 and later |
|---|
This specifies the type of data to be stored. The supported values refer to an older version of the perfSONAR software where different data types were stored in different archives. Even if the data ultimately ends up in the same archive you need to define multiple <measurement_archive> directives for each type of data you plan to store. The types have the following meanings:
- perfsonarbuoy/bwctl - Throughput tests such as those initiated by BWCTL running iperf or iperf3
- perfsonarbuoy/owamp - OWAMP tests initiated by the powstream tool
- pinger - Ping tests initiated by ping, bwping or OWAMP tests initiated by bwping running OWAMP
- traceroute - Any type of test initiated by bwtraceroute
read_url Directive
| Description: | The URL where tools should query for results |
|---|
| Syntax: | read_url URL |
|---|
| Contexts: | measurement archive |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 and later |
|---|
This is the URL where tools, such as a MaDDash dashboard, will query for test results. In general, this value will be the same as the write_url if running the perfSONAR Toolkit 3.4 or newer. You may want it to be a different value if your server uses a different public address than it uses to store data, such as in a NAT environment.
write_url Directive
| Description: | The URL where tools should send results to store |
|---|
| Syntax: | write_url URL |
|---|
| Contexts: | measurement archive |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3, ignored in 3.4, supported again in 3.5 or later |
|---|
This is the URL where tools, such as regular-testing on the perfSONAR Toolkit, should send results. This value is only useful if your measurement archive is using IP authentication. if using API key authentication, you will need to define the measurement archive in your local regulartesting.conf file. This is because it is not safe for the MeshConfig to share login credentials in it’s current form and would be difficult to manage in a large mesh.
Defining Test Topology
<group> Directive
| Description: | Describes which tests should be run between a given set of addresses. The NAME tag is used to identify the group elsewhere in the configuration. |
|---|
| Syntax: | <group NAME>...</group> |
|---|
| Contexts: | top level |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.3 or later |
|---|
The group directive is one of the primary elements used for defining your configuration. Each group has a at a minimum a type and a list of members. The combination of these elements defines which tests are run. Each member of a group must reference either an address defined in a host block or a host_class. For example, the following defines a group named example_group where each host tests to every other host in the list (i.e. type is mesh). The first two addresses in the list are explicitly defined (10.0.1.1 and 10.0.1.2) and the third is a host class (host_class::ten_gige):
<group example_group>
type mesh
member 10.0.1.1
member 10.0.1.2
member host_class::ten_gige
</group>
For more on the different types of groups, see the group type section.
type Directive
| Description: | The type of group, which further determines which options should be used in the rest of the group directive. |
|---|
| Syntax: | type disjoint|mesh|ordered_mesh|star |
|---|
| Contexts: | group |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
The type can be one of the following values:
disjoint - This type of test defines that tests are performed between a set of addresses in group A and a second set of addresses in group B. It is possible for an address to be in both groups. See a_member Directive, b_member Directive and no_agent Directive for more information on defining members. Example:
<group example_disjoint_group>
type disjoint
a_member 10.0.1.1
a_member 10.0.1.2
b_member 10.0.2.1
b_member 10.0.2.2
</group>
mesh - This type of test defines that tests are performed between all addresses in the given list. See member Directive and no_agent Directive for more information on defining members. Example:
<group example_mesh_group>
type mesh
member 10.0.1.1
member 10.0.1.2
member host_class::ten_gige
</group>
ordered_mesh - A special type of group were the order an address is listed matters. The first address is responsible for testing to all the hosts listed below it, the second host is responsible for testing to all the hosts below that, etc until the last address is reached, which does not initiate any tests. This ultimately leads to a full mesh, with hosts toward the top of the list taking on a larger burden for initiating and storing tests. This may be desirable if you have a set of more powerful hosts (in terms of hardware) you can put toward the top of this list with less powerful hosts toward the bottom. See member Directive for more information on defining members. Example:
<group example_ordered_group>
type ordered_mesh
member 10.0.1.1
member 10.0.1.2
member 10.0.2.1
member 10.0.2.2
</group>
star - A mesh where a single center address tests to all other members of the group. See center_address Directive and member Directive for more information on defining members. Example:
<group example_star_group>
type star
center_address 10.0.0.1
member 10.0.1.1
member 10.0.1.2
member 10.0.2.1
member 10.0.2.2
</group>
Note
The functional equivalent of a star group can alternatively be defined as a disjoint group where the a_member OR b_member is the center_address.
a_member Directive
| Description: | For disjoint type groups, defines an address belonging to group A. This address will only test to addresses defined in group B using the b_member directive and will NOT test to other addresses in group A (unless they are also in group B). Note that the address MUST map to an address defined in one (and only one) host directive. |
|---|
| Syntax: | a_member ADDRESS |
|---|
| Contexts: | group where type is disjoint |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.3 or later |
|---|
b_member Directive
| Description: | For disjoint type groups, defines an address belonging to group B. This address will only test to addresses defined in group A using the a_member directive and will NOT test to other addresses in group B (unless they are also in group A). Note that the address MUST map to an address defined in one (and only one) host directive. |
|---|
| Syntax: | b_member ADDRESS |
|---|
| Contexts: | group where type is disjoint |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.3 or later |
|---|
center_address Directive
| Description: | For star type groups, defines the address that will test to all other addresses defined by the member directive. Note that the address MUST map to an address defined in one (and only one) host directive. |
|---|
| Syntax: | center_address ADDRESS |
|---|
| Contexts: | group where type is star |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
member Directive
| Description: | Defines an address to be used in a group of various types (see Context row later in this table). Note that the address MUST map to an address defined in one (and only one) host directive. |
|---|
| Syntax: | member ADDRESS |
|---|
| Contexts: | group where type is mesh, ordered_mesh or star |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.3 or later |
|---|
no_agent Directive
| Description: | Defines an address that will not initiate tests when used in this group. This will override the no_agent field specified in the host directive if defined. It is recommended you use the host directive to define this if a address cannot initiate tests for any group. Only use this form if you want a host to initiate tests when used in some groups but not others. |
|---|
| Syntax: | no_agent ADDRESS |
|---|
| Contexts: | group |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.3 or later |
|---|
Dynamic Mesh Generation
tag Directive
| Description: | A custom string used to label an organization, site, host or address |
|---|
| Syntax: | tag TAG |
|---|
| Contexts: | organization, site, host, address |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.5 or later |
|---|
The tag directive labels the parent object in a custom way. This label can then be used in a <host_class> Directive definition to match items sharing the same tag. A good example is if you want to tag some <address> directives as “latency” and others as “throughput” on a dual-homed host. You can then define a host tag that selects only the “throughput” interfaces for throughput tests and “latency” interfaces for latency tests.
<host_class> Directive
| Description: | Defines a set of criteria that if a host meets, then will be included in this class. the class can then be referenced by name in test definitions to include groups of hosts. |
|---|
| Syntax: | <host_class>...<host_class> |
|---|
| Contexts: | top level |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.5 or later |
|---|
The <host_class> structure is one of the more complex in the MeshConfig. It is the foundational element in generating dynamic lists of hosts. A host_class has a name used to identify it, one or more data sources that contain an initial list of hosts, and a set of filters used to select hosts from the data sources that meet certain criteria. Optionally it may also have host properties used to set attributes on matching hosts such as measurement archives for storing test results. This host class can then be referenced in a <group> Directive as a member using the notation host_class::NAME where NAME is the name of the hos class. For example, below we define a host_class that matches all hosts in our current mesh file (that’s our data source in this case) belonging to organization Acme:
<host_class>
name acme_org
<data_source>
type current_mesh
</data_source>
<match>
<filter>
type organization
description Acme
</filter>
</match>
</host_class>
We could then letter reference this host_class in a group as follows:
<group example_group>
type mesh
member 10.0.1.1
member 10.0.1.2
member host_class::acme_org
</group>
An hosts we add to our mesh in the future with organization “Acme” will automatically get added to the test definitions using this group.
name Directive
| Description: | Names a host_class. This is used to reference the host class in group definitions later. |
|---|
| Syntax: | name CLASSNAME |
|---|
| Contexts: | <host_class> Directive |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.5 or later |
|---|
data_source Directive
| Description: | The data source from which to build an initial list of hosts before applying filters. See <config_mesh-dynamic_sources> for information in different types of data sources and their options. |
|---|
| Syntax: | <data_source>...</data_source> |
|---|
| Contexts: | <host_class> Directive |
|---|
| Occurrences: | One or more |
|---|
| Compatibility: | 3.5 or later |
|---|
match Directive
| Description: | Contains a list of filters that must be matched. An empty match filter matches everything. The exclude directive takes precedence if they both match the same host. Within a match directive, filters of the same type have an implied OR condition and filter of a different type have an implied AND condition. |
|---|
| Syntax: | <match>...</match> |
|---|
| Contexts: | <host_class> Directive |
|---|
| Occurrences: | Zero or one |
|---|
| Compatibility: | 3.5 or later |
|---|
exclude Directive
| Description: | Contains a list of filters that if matched, exclude the host from the class. An empty exclude filter matches excludes nothing. This directive takes precedence over the match directive if they both match the same host. Within an exclude directive, filters of the same type have an implied OR condition and filter of a different type have an implied AND condition. |
|---|
| Syntax: | <exclude>...</exclude> |
|---|
| Contexts: | <host_class> Directive |
|---|
| Occurrences: | Zero or one |
|---|
| Compatibility: | 3.5 or later |
|---|
<host_properties> Directive
| Description: | Defines properties to assign hosts that match the specified class. Primarily used to set the measurement archives where test results will be stored. This is a host directive, so any sub-directives supported by host are also supported by host_properties. You do not need to set the address. In general, this will only contain measurement archive directives. If the properties set in the block conflict with any already set in an explicit host directive, then the configurations will be merged. For measurement archives this means that both sets of archives will be used. |
|---|
| Syntax: | <host_properties>...<host_properties> |
|---|
| Contexts: | <host_class> Directive |
|---|
| Occurrences: | Zero or one |
|---|
| Compatibility: | 3.5 or later |
|---|
Optional Descriptive Fields
<administrator> Directive
| Description: | Defines contact information for administrator of parent element |
|---|
| Syntax: | <administrator>...</administrator> |
|---|
| Contexts: | top level, organization, site, host |
|---|
| Occurrences: | Zero or more |
|---|
| Compatibility: | 3.3 and later |
|---|
This element is primarily used for informational purposes. If defined at the top-level, it is assumed it is the contact information of the person managing this mesh. It is not strictly required in any of it’s parent elements but may be useful in keeping track of the various administrators of meshes, organization, hosts and/or sites.
name Directive
| Description: | The full name of the administrator. |
|---|
| Syntax: | name ADMINISTRATOR NAME |
|---|
| Contexts: | administrator |
|---|
| Occurrences: | Exactly one per parent |
|---|
| Compatibility: | 3.3 and later |
|---|
email Directive
| Description: | The full name of the administrator. |
|---|
| Syntax: | email ADMIN@MYDOMAIN |
|---|
| Contexts: | administrator |
|---|
| Occurrences: | Exactly one per parent |
|---|
| Compatibility: | 3.3 and later |
|---|
<location> Directive
| Description: | A directive to describe the location of the parent |
|---|
| Syntax: | <location>...</location> |
|---|
| Contexts: | organization, site, host |
|---|
| Occurrences: | Zero or One |
|---|
| Compatibility: | 3.3 or later |
|---|
This element’s intent is primarily for uses for displays that are capable of showing location information. At this point it is largely unused and may be skipped if so desired.
street_address Directive
| Description: | The street address (e.g. 1 Cyclotron Road) of the parent location |
|---|
| Syntax: | street_address ADDRESS |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
city Directive
| Description: | The city (e.g. Berkeley) of the parent location |
|---|
| Syntax: | city CITY |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
state Directive
| Description: | The state, province or other country-specific region (e.g. CA) of the parent location |
|---|
| Syntax: | state STATE |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
country Directive
| Description: | The 2-letter ISO country code (e.g. US) of the parent location |
|---|
| Syntax: | country COUNTRY |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
postal_code Directive
| Description: | The postal code (e.g. 94720) of the parent location |
|---|
| Syntax: | postal_code POSTAL_CODE |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
latitude Directive
| Description: | The latitude (e.g. 37.8717) of the parent location |
|---|
| Syntax: | latitude LATITUDE |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
longitude Directive
| Description: | The longitude (e.g. -122.2728) of the parent location |
|---|
| Syntax: | longitude LONGITUDE |
|---|
| Contexts: | location |
|---|
| Occurrences: | Exactly one |
|---|
| Compatibility: | 3.3 or later |
|---|
