Difference between revisions of "ZenPack:Microsoft Azure"

From Zenoss Wiki
Jump to: navigation, search
Line 5: Line 5:
 
|Flavor=commercial
 
|Flavor=commercial
 
|Releases={{Release
 
|Releases={{Release
|Version=1.1.0
+
|Version=1.1.1
|Release date=2016/06/08
+
|Release date=2016/06/13
 
|Compatible with=Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
 
|Compatible with=Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
 
|Requires=PythonCollector
 
|Requires=PythonCollector
Line 369: Line 369:
  
 
== Changes ==
 
== Changes ==
 +
 +
;1.1.1
 +
* Fix ID processing for Resource Manager deployment model
  
 
;1.1.0
 
;1.1.0

Revision as of 11:35, 13 June 2016


Note: This ZenPack is available in commercial versions of Zenoss. Click here to request more information about this commercial ZenPack. Click here to see all commercial ZenPacks.

Organization
Zenoss, Inc.
ZenPack name
ZenPacks.zenoss.Microsoft.Azure


Applications Monitored: 



Microsoft Azure ZenPack

The Microsoft Azure ZenPack allows you to monitor Azure Subscription managing services.

Warning

The ZenPack Catalog has moved to its new home at https://www.zenoss.com/product/zenpacks as of January 17, 2017. The following information may be out of date, and this page will eventually be removed.

Support

This ZenPack is included with commercial versions of Zenoss and enterprise support for this ZenPack is provided to Zenoss customers with an active subscription.

Releases

Version 1.1.1- Download
Released on 2016/06/13
Requires PythonCollector ZenPack
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x
Version 1.0.3- Download
Released on 2015/10/22
Requires PythonCollector ZenPack
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x
Version 1.0.2- Download
Released on 2014/02/04
Requires PythonCollector ZenPack
Compatible with Zenoss Resource Manager 4.1.x, Zenoss Resource Manager 4.2.x

Background

This ZenPack provides support for monitoring Microsoft Azure Services. Monitoring for the following Azure Subscription entities is provided using Windows Azure SDK for Python.

Bulbgraph.png Note: Please restart zenoss after installing this ZenPack to be able to add a new Azure Subscription. When creating Disk components not related to any Instance or Cloud component, they will be displayed in the Blobs table. Note that the commercial zenoss version requires ZenPacks.zenoss.Impact, otherwise the DynamicServices panel will be missing.

Gallery

Features

The features added by this ZenPack can be summarized as follows. They are each detailed further below.

  • Discovery of Azure Subscription entities.
  • Monitoring of Storage Service and Site components.
  • Event management and monitoring for certain Azure states.
  • Optional service impact with addition of Zenoss Service Dynamics product.

Discovery

The following entities will be automatically discovered through an account name, subscription ID and certificate file path you provide. The attributes, tags and collections will be updated on Zenoss' normal remodeling interval which defaults to every 12 hours.

Hosted Services (Cloud Services in Azure Management Portal)

Attributes: ID, URL, Service status, Affinity group, Location, Creation date, Modification date
Collections: Instances, Disks

Instances (Virtual Machines in Azure Management Portal)

Attributes: ID, Hosted service, Instance status, IP address, Power state, Error code
Tags: Name
Collections: Disks
Disks
Attributes: ID, Hosted service, Instance, Location, Link, Size
Tags: Name

Storage Services (Storages in Azure Management Portal)

Attributes: ID, URL, Service status, Affinity group, Location
Collections: Containers, Blobs, Tables, Queues
Containers
Attributes: ID, Storage service, URL, Modification date
Tags: Name
Collections: Blobs
Blobs
Attributes: ID, Storage service, Container, URL, Modification date, Content length, Content type, Blob type, Lease status
Tags: Name
Tables
Attributes: ID, Storage service, Updating date
Tags: Name
Queues
Attributes: ID, Storage service, URL
Tags: Name

Virtual Networks (Networks in Azure Management Portal)

Attributes: ID, Affinity group, State
Collections: Subnets
Subnets
Attributes: ID, Virtual Network, Address prefix
Tags: Name
Web Spaces
Attributes: ID, Service status, Plan, Geolocation, Workers, Availability state, Compute mode
Collections: Sites

Sites (Web Sites in Azure Management Portal)

Attributes: ID, Web space, State, Availability state, Runtime availability state, Admin enabled, Compute mode, Host names, Enabled host names, Link, Usage state
Tags: Name
Locations
Attributes: ID
Affinity Groups
Attributes: ID, Description, Location, Capabilities

Monitoring

The following metrics will be collected every 5 minutes by default. The Average statistic is collected, and the graphed value is per second for anything that resembles a rate.

Storage Services
Metrics: Blobs container count, Blobs object count, Blobs capacity
Sites
Metrics: CPU time, Filesystem storage, Memory usage, Bytes received, Bytes sent, Local bytes read, Local bytes written
Subscription
Metrics: Max core count, Max storage accounts, Max hosted services, Current core count, Current hosted services, Current storage accounts

Selective monitoring

If property zAzureMonitoringIgnore filled with proper Python expression that returns boolean value then this expression will be evaluated against each device component. And if result of that expression is True then component will not be monitored. Following variables are defined for each component and can be used inside user-defined expressions:

  • cls - component's class name
  • cmp - component object

Do not monitor all Azure blobs:

  cls == 'AzureBlob'

Do not monitor blob if its name starts with 'qa-' prefix:

  cls == 'AzureBlob' and cmp.name().startswith('qa-')

Event monitoring

According to the following Azure states this ZenPack will trigger info or error events with an appropriate summary message. Note that events will not be generated for Table and Queue Components as they are application level objects and could cause performance issues.

Subscription
Info event:
  • Current core count is equal to Max core count,
  • Current storage accounts number is equal to Max storage accounts number,
  • Current hosted services number is equal to Max hosted services number.

Hosted Service (Cloud Service)

Warning event: Hosted service is Stopped or Unavailable
Info event: service status is one of the following: Deleting, Deleted, Changing, ResolvingDns

Instance (Virtual Machine)

Warning event: Instance is Stopped or Unavailable
Info event: instance status is DeletingVM
Error event:
  • instance status is one of the following: RoleStateUnknown, FailedStartingRole, FailedStartingVM, UnresponsiveRole,
  • instance error code is one of the following: WaitTimeout, VhdTooLarge, AzureInternalError,
  • instance status is Unknown.
Disk
Warning event: Disk is Deleted or Unavailable
Error event: is_corrupted property is True
Storage Service
Warning event: Storage Service is Deleted or Unavailable
Info event: service status is one of the following: Deleting, Deleted, Changing, ResolvingDns

Virtual Network Site (Network)

Warning event: Virtual Network is Deleted or Unavailable
Info event: state is Deleting
Web Space
Error event: availability or service status is Limited
Site
Warning event: Site is Stopped, Deleted or Unavailable
Error event:
  • usage state is Exceeded,
  • availability state is Limited

Service Impact

When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact capability for services running on Microsoft Azure Service. The following service impact relationships are automatically added. These will be included in any services that contain one or more of the explicitly mentioned entities.

Service Impact Relationships
  • Subscription failure affects hosted service, storage sevice, virtual network, web space, affinity group or location.
  • Hosted service and disk failure affects related instance.
  • Storage service failure affects related container, table or queue.
  • Container failure affects related blob.
  • Web space failure affects related site.
  • Virtual network failure affects related subnet.

Billing Charts

If zProperties zAzureEAAccessKey and zAzureEAEnrollmentNumber are filled with correct values and AzureEABillingDataSourcePlugin data source plugin is attached to device, then billing charts will be available at:

Device Overview Page

Five charts on the device overview page:

  • Total cost bullet chart. It shows total money spend in current month, and also minimum, maximum and average for all previous months
  • Pie chart 'Costs by Service'. Shows distribution of money spends in current month by services.
  • Pie chart 'Costs by Region'. Shows distribution of money spends in current month by regions.
  • 'Services by top Regions'. Shows top 10 most expensive services in current month by regions.
  • 'Regions by top Services'. Shows top 10 most expensive regions in current month by services.

Billing Charts Page

Two charts on the Billing Charts page

  • Services usage trends.
  • Regions usage trends

By default values displayed on the daily basis for each Region and Service for the last 30 days. Following options can be changed by controls on the top of the page:

  • Start date - the date from data are displayed
  • End date - the date till data on graphs are displayed
  • Department - the department for which data are displayed
  • View - on which basis (daily/monthly) data are displayed.

Billing Charts for Components

For components 'Storage Services' and 'Hosted Services' choosing display option 'Billing Information' displays usage graph for particular selected component.

Usage

Adding Azure Subscriptions

Use the following steps to start monitoring Azure Subscription using the Zenoss web interface.

  1. Navigate to the Infrastructure page.
  2. Choose Add Azure Subscription from the add device button.
  3. Enter your Azure account name, subscription ID and certificate file name along with absolute path (e.g. /home/zenoss/Downloads/MSAzure.pem).
  4. Optionally choose a collector other than the default localhost.
  5. Click Add.

Enabling billing data collection for Enterprise Accounts

This requires an Access Key and Enrollment Number that can be obtained at Azure Enterprise Portal (http://ea.azure.com/)

  • Fill in zProperties zAzureEAAccessKey and zAzureEAEnrollmentNumber with values from Azure Enterprise Portal
  • Add AzureEABillingDataSourcePlugin data source plugin to monitoring template /Azure/AzureSubscription.

Prerequisites

Certificate file
To add a new Azure Subscription you will need a certificate file used in Management Certificates on Azure Management Page (see [1]), which can be stored anywhere on the disk with read permissions enabled.
If you do not have a certificate, please see Windows Azure Documentation page.
Monitoring configuration
Data monitoring and events triggering will only work if monitoring configuration on Azure Management Page (see [2]) is enabled with at least 1-2 days retention.
Network access to Azure endpoints
Your Zenoss collector should be able to establish HTTP and HTTPS (ports 80 and 443) connections to following addresses:
  • *.blob.core.windows.net
  • *.queue.core.windows.net
  • *.table.core.windows.net
  • *.servicebus.windows.net
  • management.core.windows.net

Zenoss 5.x

On Zenoss 5.x certificate file should be available inside a container. Good option is to store PEM file in DFS storage directory on 5.X:

  /opt/serviced/var/volumes/$SERVICE_ID/var-zenpacks

So inside the container, the pem file can be found here:

  /var/zenoss

Example:

1. In 5.X master host, place the azure certificate pem file in the following directory and give the pem file read permission:

  SERVICE_ID=$(serviced service list | awk '/Zenoss./{print $2}')
  cd /opt/serviced/var/volumes/$SERVICE_ID/var-zenpacks
  chmod +r azure.pem

2. On the Azure device, for "Certificate file name" provide the path to the pem file.

  /var/zenoss/azure.pem

Troubleshooting

If you encounter any issues with Zenpack instalation, please make sure that you have restarted zenoss after installation and your target disk has enough free memory and read permisions enabled. If device adding failed, try to restart your browser and make sure that you have specified a correct id and path to the certificate file (e.g. /home/zenoss/Downloads/MSAzure.pem). In case of any issues when uninstalling this Zenpack, please remove all devices first.

Note that device monitoring requires zenpython deamon to be constantly running. If you have just enabled monitoring, try pressing Zoom in to see the graphs. In case of any issues you can run zenpython daemon in foreground with zenpython run -v10 -c command and check if there are no exceptions. If zenpython runs without errors and prints monitoring data to stdout, then you should check if monitoring templates are bound to subscription and it's components.

If device can not be modeled or monitored with reason 'Server failed to authenticate the request' it may be caused by wrong system time.

Billing Charts

Billing data may have up to a five day lag due to Azure data processing delays. So if there are no any billable resources in Azure older than five days, then billing charts might be empty.

Limitations

Due to limitations of REST API this ZenPack does not monitor some of Azure services:

  • Mobile services
  • SQL Databases
  • SQL Reporting
  • Media services
  • Active Directory

Please also note, that this ZenPack does not provide neither modeling nor monitoring for Azure resources created in the Resource Manager deployment model. (Azure Resource Manager vs. classic deployment: Understand deployment models and the state of your resources).

For Virtual Machines there is no performance information available via REST API, so this ZenPack does not provide such information. Hovewer, this information can be retrieved by adding Azure VM as Linux/Windows server device in Zenoss.

For Storage Services, the graphs are updated on a daily basis. The entities are stored in $MetricsCapacityBlob each day. (MSDN)

For Sites, the graphs are updated with some delay. (MSDN)

Known bugs:


Installed Items

Installing this ZenPack will add the following items to your Zenoss system.

Device Classes
  • /Azure
Modeler Plugins
  • AzureCollector
Datasource Types
  • AzureDataSource
  • AzureEABillingDataSourcePlugin
Monitoring Templates
  • AzureSite (in /Azure)
  • AzureStorageService (in /Azure)
  • AzureSubscription (in /Azure)
Device Types
  • AzureSubscription (in /Azure)
Component Types
  • AzureHostedService (on AzureSubscription)
  • AzureInstance (on AzureHostedService)
  • AzureDisk (on AzureInstance)
  • AzureStorageService (on AzureSubscription)
  • AzureContainer (on AzureStorageService)
  • AzureQueue (on AzureStorageService)
  • AzureTable (on AzureStorageService)
  • AzureBlob (on AzureContainer)
  • AzureVirtualNetworkSite (on AzureSubscription)
  • AzureSubnet (on AzureVirtualNetworkSite)
  • AzureWebSpace (on AzureSubscription)
  • AzureSite (on AzureWebSpace)
  • AzureLocation (on AzureSubscription)
  • AzureAffinityGroup (on AzureSubscription)

Changes

1.1.1
  • Fix ID processing for Resource Manager deployment model
1.1.0
  • Add billing data collection and charting for Azure Enterprise accounts
  • Add possibility to skip monitoring of particular components
1.0.3
  • 5.x support.
  • Monitoring performance improved.
  • Various bug fixes.

Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.Microsoft.Azure-*.egg
  4. Restart these services:
    $ zenoss restart


Discuss

Purplemarker.png New: Don't forget to add yourself to the Zenoss User Map!

blog comments powered by Disqus



Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.Microsoft.Azure-*.egg
  4. Restart these services:
    $ zenoss restart


Discuss

Purplemarker.png New: Don't forget to add yourself to the Zenoss User Map!

blog comments powered by Disqus