ZenPack:OpenStack (Provider View)

From Zenoss Wiki
Revision as of 14:48, 29 March 2017 by Jwilmes (Talk | contribs)$7

Jump to: navigation, search
Organization
Zenoss, Inc.
License
GNU General Public License, Version 2, or later
ZenPack name
ZenPacks.zenoss.OpenStackInfrastructure
More Information
GitHub page/HomePage
Git sources (for cloning)
Link


Applications Monitored: 



OpenStack (Provider View) ZenPack

This ZenPack allows for monitoring of OpenStack from a service provider perspective. This means that in addition to the user-oriented components supported in the regular OpenStack ZenPack (instances, flavors, images), the underlying OpenStack servers and software are monitored.

Warning

The ZenPack Catalog has moved to its new home at https://www.zenoss.com/product/zenpacks as of January 17, 2017. The following information may be out of date, and this page will eventually be removed.

Support

This is an Open Source ZenPack developed by Zenoss, Inc. Enterprise support for this ZenPack is available to commercial customers with an active subscription.

Releases

Version 2.3.3- Download
Summary of changes: Fixes issues with modeling certain openstack environments.
Released on 2017/03/29
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack,Linux Monitor ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Core 5.1.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x, Zenoss Resource Manager 5.x.x
Incompatible with Zenoss Core 2.5.x, Zenoss Core 3.1.x, Zenoss Core 3.2.x, Zenoss Resource Manager 4.1.x
Version 2.3.2- Download
Summary of changes: fixed NotFound exception in the zenpack by wrapping brain.getObject() into a try/except block.
Released on 2016/12/19
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack,Linux Monitor ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Core 5.1.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x, Zenoss Resource Manager 5.x.x
Version 2.3.1- Download
Summary of changes: .
  • Upgrade txsshclient for changes in twisted.conch
Released on 2016/11/03
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack,Linux Monitor ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Core 5.1.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
Version 2.3.0- Download
Summary of changes: .
  • Added support for Mitaka.
  • Provide various host-checking fixes
  • Add publicURL if adminURL not available
  • Upgrade ZenPackLib to 1.1.0
Released on 2016/09/28
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack,Linux Monitor ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Core 5.1.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
Version 2.2.0- Download
Summary of changes: Add Cinder block storage components; add LVM, Ceph block storage integration via LinuxMonitor and Ceph ZenPacks; and various bug fixes
Released on 2016/07/05
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack,Linux Monitor ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Core 5.1.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
Version 2.1.3- Download
Summary of changes: Allow for broken hostname by F5-lbaas-agent; remove deprecated Ceilometer Agent Hearbeat notifications.
Released on 2016/02/26
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Core 5.0.x, Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x
Version 2.0.0
Summary of changes: Initial Version
Released on 2014/10/09
Requires PythonCollector ZenPack,OpenStack (Tenant View) ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Resource Manager 4.2.x
Enhances OpenStack Cloud Monitor

Background

This ZenPack allows for monitoring of OpenStack from a service provider perspective. This means that in addition to the user-oriented components supported in the regular OpenStack ZenPack (instances, flavors, images), the underlying OpenStack servers and software are monitored.

Features

  • Monitors Overall OpenStack state
  • Monitors Nova, Neutron and Cinder health
  • Monitors multiple Tenant Infrastructure
  • Models and monitors Nova Services and components
  • Models and monitors Neutron Agents and components
  • Models and monitors Cinder Services and components
  • Impact and Root-Cause (Requires Zenoss Service Dynamics)

Gallery

Prerequisites

  • OpenStack (Icehouse or newer)
  • ceilometer_zenoss compatible with your version of OpenStack
    • ceilometer_zenoss 1.0.1 for Kilo and prior versions (https://github.com/zenoss/ceilometer_zenoss)
    • ceilometer_zenoss 1.1.0 for Liberty and newer versions
    • This Zenoss-specific plugin must be installed on all your Ceilometer nodes (see below)
    • If not installed and configured properly, Instances and vNICs graphs will be blank, and Zenoss will not detect changes (such as new instances or instance state changes) until a full model is performed.
  • PyYAML needs to be installed in order to use the YAML configuration
    • All 5.X versions of Zenoss have PyYAML already installed
    • Only 4.2.4 SP763 or 4.2.5 SP378 or earlier affected
    • You will have to add this Python package to your Zenoss installation otherwise
    • Update to the latest RPS or "easy_install pyyaml" manually if required.
  • SSH credentials to the Linux devices in your OpenStack environment
    • Configure the zCommandUsername/zCommandPassword/zKeyPath properties on the /Devices/Server/SSH/Linux/NovaHost device class.
    • If your Linux devices are already managed under Zenoss, move them into /Devices/Server/SSH/Linux/NovaHost
  • Administrative credentials for your OpenStack environment
    • Username, password, keystone URL, Region

Installed Items

Configuration Properties
  • zOpenStackAuthUrl: The URL of the Identity endpoint.
  • zOpenStackCeilometerUrl: The URL to the Ceilometer API.
  • zOpenStackExtraHosts: The list of extra hosts that will be added to the system once OpenStack Infrastructure device is modeled.
  • zOpenStackHostDeviceClass: Used as a default device class for defined hosts in zOpenStackExtraHosts and zOpenStackNovaApiHosts properties. Default is /Server/SSH/Linux/NovaHost.
  • zOpenStackNovaApiHosts: The list of hosts upon which nova-api runs. This is required when the IP address in the nova API url does not match any known host.
  • zOpenStackCinderApiHosts: The list of hosts upon which cinder-api runs. This is required when the IP address in the cinder API url does not match any known host.
  • zOpenStackHostMapToId: A list of <name>=<id>, used to force a host referred to by openstack with the given name to be represented in Zenoss as a host component with the given ID. (this is not commonly used)
  • zOpenStackHostMapSame: A list of <name1>=<name2>, used to inform the modeler that the same host may be referred to with a different name by some part of openstack. (this is not commonly used)
  • zOpenStackNeutronConfigDir: The path where to search for Neutron's configs. Default is /etc/neutron.
  • zOpenStackProjectId: Corresponds to tenant name, project to work on.
  • zOpenStackRegionName: The name of OpenStack Region to use. Regions are autonomous OpenStack clouds joined together through shared Keystone identity server and managed through common interface.
  • zOpenStackRunNovaManageInContainer, zOpenStackRunVirshQemuInContainer, zOpenStackRunNeutronCommonInContainer: Used for container based OpenStack installation. Leave them empty.
Device Classes
  • /OpenStack: Root OpenStack device class. Typically devices should not be put in this device class.
  • /OpenStack/Infrastructure: Device class for OpenStack Infrastructure endpoints.
  • /Server/SSH/Linux/NovaHost: Device class for Nova host instances.
Modeler Plugins
  • zenoss.OpenStackInfrastructure: Used to get component information by OpenStack API clients.
  • zenoss.cmd.linux.openstack.hostfqdn: Used to get OpenStack host FQDN.
  • zenoss.cmd.linux.openstack.inifiles: Used to gather neutron.conf and ml2_conf.ini files.
  • zenoss.cmd.linux.openstack.libvirt: Used to get OpenStack instance virtual NIC information using libvert.
  • zenoss.cmd.linux.openstack.nova: Used to get OpenStack Nova API version.
Datasources
  • CinderServiceStatusDataSource: Used to check the status of Cinder services via the Cinder API.
  • EventsAMQPDataSource: Used to capture data and events from OpenStack Ceilometer via AMQP.
  • HeartbeatsAMQPDataSource: Used to capture heartbeats from OpenStack Ceilometer via AMQP.
  • NeutronAgentStatusDataSource: Used to check the status of Neutron agents via the Neutron API.
  • NovaServiceStatusDataSource: Used to check the status of Nova services via the Nova API.
  • PerfAMQPDataSource: Used to capture data and events from OpenStack Ceilometer via AMQP.
  • PerfCeilometerAPIDataSource: Used to capture datapoints from OpenStack Ceilometer.
  • QueueSizeDataSource: Used to check the number of messages in Ceilometer queues.
Monitoring Templates
  • /OpenStack/Infrastructure/
    • Endpoint
    • Instance
    • vNIC
Event Classes and Mappings
  • /OpenStack
    • OpenStack Events Default
  • /OpenStack/Cinder
  • /OpenStack/Cinder/Snapshot
    • Cinder Snapshot default mapping
  • /OpenStack/Cinder/Volume
    • cinder.volume default mapping
  • /OpenStack/compute
  • /OpenStack/compute/instance
    • compute.instance default mapping
    • compute.instance.create.error
    • compute.instance.exists
    • compute.instance.exists.verified.old
  • /OpenStack/dhcp_agent
    • dhcp_agent defaultmapping
  • /OpenStack/firewall
    • firewall defaultmapping
  • /OpenStack/firewall_policy
    • firewall_policy defaultmapping
  • /OpenStack/firewall_rule
    • firewall_rule defaultmapping
  • /OpenStack/floatingip
    • floatingip defaultmapping
  • /OpenStack/network
    • network defaultmapping
  • /OpenStack/port
    • port defaultmapping
  • /OpenStack/router
    • router defaultmapping
  • /OpenStack/security_group
    • security_group defaultmapping
  • /OpenStack/security_group_rule
    • security_group_rule defaultmapping
  • /OpenStack/subnet
    • subnet defaultmapping
  • /Status/Heartbeat/
    • openStackCeilometerHeartbeat
  • /Status
    • openStackCinderServiceStatus
    • openStackIniFileAccess
    • openStackIniFileOptionParsing
    • openStackNeutronAgentStatus
    • openStackNovaServiceStatus
Processes
  • /OpenStack
  • /OpenStack/ceilometer-agent-central
  • /OpenStack/ceilometer-agent-compute
  • /OpenStack/ceilometer-agent-notification
  • /OpenStack/ceilometer-alarm-evaluator
  • /OpenStack/ceilometer-alarm-notifier
  • /OpenStack/ceilometer-api
  • /OpenStack/ceilometer-collector
  • /OpenStack/ceilometer-polling
  • /OpenStack/cinder-api
  • /OpenStack/cinder-backup
  • /OpenStack/cinder-scheduler
  • /OpenStack/cinder-volume
  • /OpenStack/glance-api
  • /OpenStack/glance-registry
  • /OpenStack/gnocchi-metricd
  • /OpenStack/gnocchi-statsd
  • /OpenStack/keystone-admin
  • /OpenStack/keystone-all
  • /OpenStack/keystone-main
  • /OpenStack/neutron-dhcp-agent
  • /OpenStack/neutron-l3-agent
  • /OpenStack/neutron-lbaas-agent
  • /OpenStack/neutron-metadata-agent
  • /OpenStack/neutron-metering-agent
  • /OpenStack/neutron-openvswitch-agent
  • /OpenStack/neutron-server
  • /OpenStack/nova-api
  • /OpenStack/nova-cert
  • /OpenStack/nova-compute
  • /OpenStack/nova-conductor
  • /OpenStack/nova-consoleauth
  • /OpenStack/nova-network
  • /OpenStack/nova-scheduler
  • /OpenStack/rabbitmq-server

Basic Usage

Bulbgraph.png Note: Full Modeling

The OpenStackInfrastructrue ZenPack models vNICs associated with OpenStack Instances. In order to correctly model these vNICs, you must first fully model the OpenStack environment and then configure and model the /Server/SSH/Linux/NovaHost devices. See the section below for details on configuration specifics.

Device Setup via UI

Ambox warning.jpeg Warning: Host Setup:

We recommend against manually configuring Linux host devices!

Linux Host device names must use the same FQDN hostname that Openstack ZenPack is expecting. Choosing different device names will break this process. Therefore it is STRONGLY recommended that you do not manually pre-configure those Linux devices, but rather allow the OpenStack ZenPack to automatically discover and configure those devices at model time.

If this is not possible, you will have to re-name and re-configure those hosts and place them into the /Server/SSH/Linux/NovaHost device class manually, which is very labor intensive.

Once the OpenStack ZenPack is installed and you can begin monitoring by going to the infrastructure screen and clicking the normal button for adding devices. You'll find a new option labeled, "Add OpenStack Endpoint (Infrastructure)."

Choose that option and you'll be presented with a dialog asking for the following inputs.

  1. Device To Create - non-empty, non-ip, non-dns, unique name to use for this device in Zenoss. See note below.
  2. Auth URL - A keystone URL, such as http://<hostname>:5000/v2.0/
  3. Username, Password/API Key, Project/Tenant ID - *Administrative* credentials to your Zenoss instance.
  4. Region Name - choose the correct region from the drop-down. You may only choose one, so each region you wish to manage must be registered as a separate endpoint in Zenoss.
  5. Ceilometer URL - Will auto-populate based on the other selections.

Once you click Add, Zenoss will contact the OpenStack API and discover servers, images and flavors. Once it is complete you'll find a new device in the OpenStack device class with the same name as the hostname or IP you entered into the dialog. Click into this new device to see everything that was discovered.

Bulbgraph.png Note: Device Name

The Device name should be a non-empty, non-hostname, non-IP, since that name will be used when the host is registered as a Linux device. The name should be unique within the Zenoss environment. This is especially important if you are adding another device that share the same IP address or hostname that already exist on another device. Not doing this may result in devices with the same name conflicting with each other. (e.g. attempting to model device would show modeling results that belong to another device OR device would show relations that do not belong to that device)

Device Setup via Zenbatchload

You can setup the device using *zenbatchload* as follows:

zenbatchload <filename>

where <filename> should have the form:

/Devices/OpenStack/Infrastructure loader='openstackinfrastructure', \
    loader_arg_keys=['deviceName', 'username', 'api_key', 'project_id', 'auth_url', 'ceilometer_url', 'region_name', 'collector']
<devicename> username='<username>', api_key='<password>', project_id='<tenant ID>', \
    auth_url='http://<ip address>:5000/v2.0/', region_name='RegionOne'

/Devices/Server/SSH/Linux/NovaHost zCommandUsername='username', zCommandPassword='password'

  • As mentioned before, zCommandUsername/zCommandPassword properties must be set for /Devices/Server/SSH/Linux/NovaHost devices (and vNICs) to be correctly modeled.

Organizational Elements

The following organizational elements are discovered
  • Regions
  • Availability Zones
  • Tenants
The following virtual-machine elements are discovered
  • Nova Services (processes supporting nova servers)
  • Instances (Servers)
  • Hosts
  • Hypervisors
  • Images
  • Flavors
  • Nova API Endpoints
The following network elements are discovered
  • Neutron Agents
  • Networks
  • Subnets
  • Routers
  • Ports
  • Floating-Ips
  • vNICs (from NovaHost compute device model)
The following block storage elements are discovered
  • Cinder Services (processes supporting block storage)
  • Cinder API Endpoints
  • Volumes
  • Volume Snapshots
  • Volume Types
  • Storage Pools
  • Cinder Quotas

Metrics

The following component level metrics are collected

Instances

  • CPU Utilization (percent)
  • Disk Requests (requests/sec)
  • Disk IO Rate (bytes/sec)

vNICs

  • Network Packet Rate (packets/sec)
  • Network Throughput (bytes/sec)

Hosts

  • Load Average (processes)
  • CPU Utilization (percent)
  • Free Memory (bytes)
  • Free Swap (bytes)
  • IO (sectors/sec)

Nova Services

  • CPU Utilization (percent)
  • Memory Utilization (bytes)
  • Process Count (processes)

Neutron Agents

  • CPU Utilization (percent)
  • Memory Utilization (bytes)
  • Process Count (processes)

Cinder Services

  • CPU Utilization (percent)
  • Memory Utilization (bytes)
  • Process Count (processes)

Volumes (requiring LinuxMonitor ZenPack >= 2.0.0)

  • Storage Utilization (percent)
  • Operation Throughput (operations/sec)
  • Merge Rate (merged/sec)
  • Sector Throughput (sectors/sec)
  • IO Operations (operations)
  • IO Utilization (percent)
  • Weighted IO Utilization (weighted percent)

Volume Snapshots (requiring LinuxMonitor ZenPack >= 2.0.0)

  • Storage Utilization (percent)
  • Operation Throughput (operations/sec)
  • Merge Rate (merged/sec)
  • Sector Throughput (sectors/sec)
  • IO Operations (operations)
  • IO Utilization (percent)
  • Weighted IO Utilization (weighted percent)
The following device level metrics are collected

Flavors

  • Total (count)

Images

  • Total (count)
  • Total count per image state

Servers

  • Total (count)
  • Total count per server state

Queues

  • Event (count)
  • Performance (count)

Agents

  • Total (count)
  • Total count per agent type

Networks

  • Total (count)
  • Total count per network state

Routers

  • Total (count)
  • Total count per router state

Volumes

  • Total (count)
  • Total count per volume state

Volume Snapshots

  • Total (count)
  • Total count per volume snapshot state

Volume Pool

  • Total (count)
  • Total count per volume pool state

Bulbgraph.png Note: All metrics collected by OpenStack Ceilometer may be collected and graphed in Zenoss, and all events processed through Ceilometer are exposed via the Zenoss Event Console.

Host Based Installation vs Container Based Installation

This zenpack supports both host based installation and container based installation.

Host Based Installation

With host based installation, the OpenStack binaries are directly installed on the host. For instance, the binary nova-api will be installed directly in the controller host's /usr/bin directory. In this case, edit the Configuration Properties of the /Server/SSH/Linux/NovaHost devices, zOpenStackRunNovaManageInContainer, zOpenStackRunVirshQemuInContainer, and zOpenStackRunNeutronCommonInContainer, after modeling the OpenStackInfrastructure device, but before modeling the /Server/SSH/Linux/NovaHost devices, and make sure their values are empty:

      zOpenStackRunNovaManageInContainer
      zOpenStackRunVirshQemuInContainer
      zOpenStackRunNeutronCommonInContainer

Container Based Installation

With container based installation, each OpenStack binary runs inside its own container. In this case, the Configuration Properties of the /Server/SSH/Linux/NovaHost devices, zOpenStackRunNovaManageInContainer, zOpenStackRunVirshQemuInContainer, and zOpenStackRunNeutronCommonInContainer, should be set to the container names, before modeling Nova hosts, as:

      zOpenStackRunNovaManageInContainer novacompute
      zOpenStackRunVirshQemuInContainer novacompute
      zOpenStackRunNeutronCommonInContainer neutron_server

Ceilometer Enablement

Although you may add an OpenStack device to Zenoss, as shown above, event and performance data will not be collected until the following steps are performed.

Zenoss Configuration Steps

On the Zenoss host in Zenoss 4.X, or RabbitMQ container in Zenoss 5.X:

  • ssh in as the Zenoss user:
  • export ZPDIR=$ZENHOME/ZenPacks/ZenPacks.zenoss.OpenStackInfrastructure*
  • Now execute:
   $ZPDIR/ZenPacks/zenoss/OpenStackInfrastructure/bin/openstack_amqp_config

This script will create a new user in RabbitMQ and set up its permissions. It will print out the proper values to use for amqp_hostname, amqp_port, amqp_userid, amqp_password, and amqp_virtualhost during ceilometer configuration (see below)

OpenStack Ceilometer Configuration Steps

Zenoss relies upon a Ceilometer dispatcher plugin to ship raw event and metering data from Ceilometer to Zenoss for storage in the Zenoss event and performance databases. This integration is done by publishing messages to Zenoss's RabbitMQ server.

This dispatcher should be installed on all nodes running any ceilometer, but particularly those running ceilometer-collector or ceilometer-agent-notification.

To install ceilometer_zenoss 1.0.1 for Kilo and prior versions of OpenStack, perform the following steps on each of these OpenStack nodes (NOT on the Zenoss collector):

  sudo pip -q install --force-reinstall https://github.com/zenoss/ceilometer_zenoss/archive/master.zip
  sudo cp /usr/lib/*/site-packages/ceilometer_zenoss/event_definitions.yaml /etc/ceilometer/

To install the stable versions of 1.1.0 for Liberty and newer versions of OpenStack, from Zenoss RPM repository download an appropriate rpm file to the OpenStack node that fits the host's OS version. Before installing RPMs, make sure the older version of cepilometer_zenoss has been erased. One can check this using:

  $ pip list | grep ceilometer_zenoss

or:

  $ rpm -qa | grep ceilometer_zenoss

To erase the old version of installed RPM:

  $ rpm -e <installed RPM>

Next install the stable versions of 1.1.0 RPM:

  $ rpm -Uvh <ceilometer_zenoss rpm>

The following configuration must be added to /etc/ceilometer/ceilometer.conf file:

  [notification]
  # Save event details.
  store_events=True
 
  [dispatcher_zenoss]
  # Device that the OpenStack environment is registered as in Zenoss.
  zenoss_device = ...
  # The port number to which the underlying TCP connection is made.
  # Usually, AMQP port is 5672.
  amqp_port = ...
  # Valid AMQP user id.
  amqp_userid = ...
  # Valid AMQP password for the above user id.
  amqp_password = ...
  # The name of the "virtual host" (or vhost) that specifies the namespace 
  # for entities (exchanges and queues) referred to by the protocol.
  amqp_virtual_host = ...
  # It must be an accessible interface from your OpenStack system,
  # and that interface must be one that accepts AMQP connections for your Zenoss installation.
  amqp_hostname = ...

For Liberty and prior versions, in the [DEFAULT] section, add the line:

  dispatcher = zenoss

For Mitaka and newer versions, in the [DEFAULT] section, add the lines:

  meter_dispatchers = zenoss
  event_dispatchers = zenoss
The order of the sections is important, the [dispatcher_zenoss] section needs to be added at the end of ceilometer.conf file.

Bulbgraph.png Note: Please restart all Ceilometer services on all hosts after making these changes.

Optional Steps

VM State Changes

By default, instance state changes will be captured by Zenoss when certain events occur, for example, when an instance is shut down, the state change to SHUTDOWN will be reflected in Zenoss.

However, certain state changes that don't correspond to another defined event may not be picked up until the next time Zenoss models the environment.

If you would like to reduce the likelihood of this occurring, you can configure OpenStack Nova to send an event (through ceilometer) to Zenoss whenever any VM state change occurs by adding the following to /etc/nova/nova.conf on all Nova hosts:

  [DEFAULT]
  notify_on_state_change=vm_and_task_state

For Liberty:

  [oslo_messaging_notifications]
  notification_driver = messagingv2

For Mitaka:

  [oslo_messaging_notifications]
  driver = messagingv2

These two entries should be under the section [vnc], not under [DEFAULT]

  [vnc]
  novncproxy_host=0.0.0.0
  novncproxy_port=6080

Save /etc/nova/nova.conf and restart nova services.

Note that notify_on_state_change will cause increased event load, both on OpenStack and Zenoss, and additional processing within the event transforms in Zenoss to keep the model consistent. Since most instance changes will still be caught without this option enabled, it is recommended to leave notify_on_state_change disabled if your OpenStack environment is very large.

Increasing Polling Interval

Zenoss will process performance datapoints from Ceilometer every 10 minutes, since by default, Ceilometer will only produce one datapoint every 10 minutes. This can be adjusted by modifying the "interval: 600" line in your pipeline.yaml file (typically /etc/ceilometer/pipeline.yaml).

Device Status

The OpenStackInfrastructure Device Status is now handled by ZenPackLib 1.1.0. The device Status is determined by OpenStackInfrastructure data source events. The device is down if a critical event is received from one of OpenStackInfrastructure's data sources. Otherwise the device status is up. Please note that being able to model OpenStackInfrastructure device is not equivalent to device status being up.

Since the older versions of OpenStackInfrastructure ZenPack processed device status differently, before upgrading from older versions of the ZenPacks, please make sure that the ceilometer enablement is configured properly, and Zenoss server receives heartbeat events from the OpenStack host.

Zenoss Analytics

This ZenPack provides additional support for Zenoss Analytics. Perform the following steps to install extra reporting resources into Zenoss Analytics after installing the ZenPack.

  1. Copy analytics-bundle.zip from $ZENHOME/ZenPacks/ZenPacks.zenoss.OpenStackInfrastructure*/ZenPacks/zenoss/OpenStackInfrastructure/analytics/ on your Zenoss server.
  2. Navigate to Zenoss Analytics in your browser.
  3. Login as superuser.
  4. Remove any existing OpenStackInfrastructure ZenPack folder.
    1. Choose Repository from the View menu at the top of the page.
    2. Expand Public in the list of folders.
    3. Right-click on OpenStackInfrastructure ZenPack folder and choose Delete.
    4. Confirm deletion by clicking OK.
  5. Add the new OpenStackInfrastructure ZenPack folder.
    1. Choose Server Settings from the Manage' menu at the top of the page.
    2. Choose Import in the left page.
    3. Remove checks from all check boxes.
    4. Click Choose File to import a data file.
    5. Choose the analytics-bundle.zip file copied from your Zenoss server.
    6. Click Import.

You can now navigate back to the OpenStackInfrastructure ZenPack folder in the repository to see the following resources added by the bundle.

Domains
  • OpenStackInfrastructure Domain
Ad Hoc Views
  • OpenStack Instance List

Domains can be used to create ad hoc views using the following steps.

  1. Choose Ad Hoc View from the Create menu.
  2. Click Domains at the top of the data chooser dialog.
  3. Expand Public then OpenStackInfrastructure ZenPack.
  4. Choose the OpenStackInfrastructure Domain domain


Service Impact and Root Cause Analysis

When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact and root cause analysis capabilities for OpenStack infrastructure and instances. The service impact relationships shown in the diagram and described below are automatically added. These will be included in any services that contain one or more of the explicitly mentioned components.

Impact Relationship Diagram

Recommended Impact Setup

Since most components will be related to Tenants and Region we recommend:

  • Navigate to Services (Impact)
  • Add a Dynamic Service to your Services tab
  • Add all Tenants to the Dynamic Service
  • Add all Regions to the Dynamic Service

Impact Relations

Component failures will affect Impact as follows:

Internal Impact Relationships
  • OpenStack API endpoint impacts all Hosts
  • Availability zone impacts associated Region
  • Host impacts associated Hypervisors, Nova Services, Cells, Nova Apis, Neutron Agents, and Cinder Services
  • Hypervisor impacts the resident Instances (VMs)
  • Nova Service affects the Availability Zone or Region that it supports
  • Instance impacts the associated Tenant
  • vNIC impacts the related Instance.
  • Port impacts associated Instance
  • Subnet impacts associated Ports and Tenants
  • Floating-IP impacts associated Port
  • Network impacts associated Subnets and Tenants
  • Router impacts associated Subnets and Floating-ips
  • Neutron Agent impacts associated Networks, Subnets and Routers
  • volume impacts Instances (VMs), Volume Snapshots
External Impact Relationships
  • Instance impacts guest operating system device.
  • Cisco UCS vNIC impacts related host's underlying Linux device NIC.
  • Cisco UCS service profile impacts host's underlying Linux device.
  • Host impacted by associated Linux device.
  • OS Processes on an underlying Linux device impact corresponding Nova APIs, Nova Services, Neutron Agents and Cinder Services on Host.

ML2 Integration

Provides linkage between OpenStackInfrastrcuture components and relevant integrated device components; imports performance graphs from relevant integrated device components; provides impact graph relations between OpenStackInfrastrcuture components and relevant integrated device components

  • OpenvSwitch Integration via OpenvSwitch ZenPack
  • APIC Integration via Cisco APIC ZenPack
  • NSX Integration via NSX ZenPack

Cinder Integration

Provides linkage between OpenStackInfrastrcuture components and relevant integrated device components; imports performance graphs from relevant integrated device components; provides impact graph relations between OpenStackInfrastrcuture components and relevant integrated device components

  • LVM logical volumes Integration via LinuxMonitor 2.0.0 ZenPack
  • Ceph RBD volumes Integration via Ceph 2.0.0 ZenPack

Tested OpenStack Releases

Version 2.0.0 was tested against Icehouse and Juno; 2.1.3 against Juno and Kilo; 2.2.0 against Juno, Kilo, Liberty; 2.3.0 against Mitaka.

Trouble Shooting

Nova Hosts

Model Device

ERROR zen.PythonClient

Error in zenoss.cmd.linux.openstack.nova: Connection was refused by other side: 111: Connection refused. In the upper left corner immediately under device name and device class, there should be an IP address. Modeling would fail if there is no IP address listed. The absence of IP address is likely a DNS issue.

Hosts are duplicated by Name and IP

The zOpenStackHostMapSame exist for mapping equivalent hosts. If you have an FQDN and IP that are listed separately make a device-level mapping entry that takes the form FQDN=IP. For example:

 my.example.com=10.100.200.222

Note that you must have DNS functioning for my.example.com to acquire its IP address and set its manageIp correctly.

Hosts are not identified to a pre-existing Linux ID

In this case there is already a Linux host identified by an ID name. Use zOpenStackHostMapToId to make a device-level equivalent hosts mapping to this ID value. For example:

 my.example.com=myID
 10.100.200.222=myID

Known Issues

  1. ZEN-14585: The same endpoint can not be monitored both as user and an infrastructure endpoints.
    1. Workaround: If you have been previously monitoring the endpoint as a User endpoint, delete the device before you re-add it as an Infrastructure endpoint.
  2. ZEN-17905: Nova APIs component: Grey icons for Enabled and State after model/monitor.
    1. OpenStack nova service API does not provide information about Nova-API, so its status is, in fact, unknown.

Changes

2.3.2
  • Wrap brain.getObject() into try/except block (ZPS-442)
2.3.1
  • Upgrade txsshclient to fix critical change in twisted.conch (ZEN-25870)
2.3.0
  • Added support for Mitaka.
  • Provide various host-checking fixes: (ZEN-24803, ZEN-25262)
  • Prevent queues from being delete when device is removed/re-added (ZEN-24803)
  • Use publicURL if adminURL not working: (ZEN-24546)
  • Upgrade ZenPackLib to 1.1.0 to fix Liberty/Mitaka status: (ZEN-24464)
2.2.0
  • Added Cinder block storage components.
  • Added LVM, Ceph block storage integration via LinuxMonitor and Ceph ZenPacks.
  • Various bug fixes
2.1.3
  • Fix malformed hostnames in the F5 LBAAS plugin (ZEN-22126)
2.1.2
  • Remove deprecated ceilometer-agent-notification heartbeats
2.1.1
  • Various bug fixes
  • Add meta.zcml feature tags for Neutron Integration
2.1.0
  • Added Neutron network components
  • Update Impact models for Neutron
  • Update multiple UI interfaces
  • Upgrade to ZenPackLib 1.0.1
  • Add ML2 Pluggin Capability


2.0.0
  • Initial Release

Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.OpenStackInfrastructure-*.egg
  4. Restart these services:
    $ zenoss restart

Developer Mode Installation

In order to do a development mode installation you will want to clone the existing git repository, and then use the --link flag with the zenpack command:

  1. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  2. Start by cloning the upstream repository:
    $ git clone https://github.com/zenoss/ZenPacks.zenoss.OpenStackInfrastructure.git
  3. Next, perform the installation:
    $ zenpack --link --install ZenPacks.zenoss.OpenStackInfrastructure
  4. Finally, restart these serivices:
    $ zenoss restart

Discuss

Purplemarker.png New: Don't forget to add yourself to the Zenoss User Map!

blog comments powered by Disqus