Difference between revisions of "ZenPack:Microsoft Windows"

From Zenoss Wiki
Jump to: navigation, search
m
m
Line 653: Line 653:
 
* The custom widget for MSSQL Server credentials is not compatible with Zenoss 4.1.x, therefore the ''zDBInstances'' property in this version should be set as a valid JSON list (e.g. ''[{"instance": "MSSQLSERVER", "user": "", "passwd": ""}]'' ).
 
* The custom widget for MSSQL Server credentials is not compatible with Zenoss 4.1.x, therefore the ''zDBInstances'' property in this version should be set as a valid JSON list (e.g. ''[{"instance": "MSSQLSERVER", "user": "", "passwd": ""}]'' ).
 
* When upgrading to version 2.2.0, you may see a segmentation fault during the install.  This occurs when upgrading from versions 2.1.3 and previous.  To ensure a successful installation, run the install once more and restart Zenoss.
 
* When upgrading to version 2.2.0, you may see a segmentation fault during the install.  This occurs when upgrading from versions 2.1.3 and previous.  To ensure a successful installation, run the install once more and restart Zenoss.
 +
* Payload encryption is not currently supported for version 2.2.0.  Please use basic authentication.
  
 
A current list of known issues related to this ZenPack can be found with [https://jira.zenoss.com/issues/?jql=%22Affected%20Zenpack(s)%22%20%3D%20MicrosoftWindows%20AND%20status%20not%20in%20(closed%2C%20%22awaiting%20verification%22)%20ORDER%20BY%20priority%20DESC%2C%20id this JIRA query]. You must be logged into JIRA to run this query. If you don't already have a JIRA account, you can [https://jira.zenoss.com/secure/Signup!default.jspa create one here].
 
A current list of known issues related to this ZenPack can be found with [https://jira.zenoss.com/issues/?jql=%22Affected%20Zenpack(s)%22%20%3D%20MicrosoftWindows%20AND%20status%20not%20in%20(closed%2C%20%22awaiting%20verification%22)%20ORDER%20BY%20priority%20DESC%2C%20id this JIRA query]. You must be logged into JIRA to run this query. If you don't already have a JIRA account, you can [https://jira.zenoss.com/secure/Signup!default.jspa create one here].

Revision as of 20:26, 19 December 2014

Organization
Zenoss, Inc.
License
GNU General Public License, Version 2, or later
ZenPack name
ZenPacks.zenoss.Microsoft.Windows
More Information
GitHub page/HomePage
Git sources (for cloning)
Link


Applications Monitored: 



Microsoft Windows ZenPack

Monitoring for Microsoft Windows servers.

Warning

The ZenPack Catalog has moved to its new home at https://www.zenoss.com/product/zenpacks as of January 17, 2017. The following information may be out of date, and this page will eventually be removed.

Support

This is an Open Source ZenPack developed by Zenoss, Inc. Enterprise support for this ZenPack is available to commercial customers with an active subscription.

Releases

Version 2.2.0- Download
Released on 2014/12/16
Requires PythonCollector ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Resource Manager 4.2.x
Version 2.1.3- Download
Released on 2014/10/27
Requires PythonCollector ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Resource Manager 4.1.x, Zenoss Resource Manager 4.2.x
Version 2.0.3- Download
Released on 2014/04/16
Requires PythonCollector ZenPack
Compatible with Zenoss Core 4.2.x, Zenoss Resource Manager 4.1.x, Zenoss Resource Manager 4.2.x

Background

This ZenPack provides support for monitoring Microsoft Windows. Monitoring is performed using the Windows Remote Management (WinRM) and Windows Remote Shell (WinRS) to collect Windows Management Instrumentation (WMI) and Perfmon data.

Bulbgraph.png Note: This ZenPack supersedes the earlier ZenPack named ZenPacks.zenoss.WindowsMonitor for Windows platforms that support WinRM. If you have ZenPacks.zenoss.WindowsMonitor installed on your system, please read the #Transitioning from WindowsMonitor section below.

Video

Gallery

Features

The features added by this ZenPack can be summarized as follows. They are each detailed further below.

  • Initial discovery and periodic remodeling of relevant components.
  • Performance monitoring.
  • Event management.

Discovery

The following components will be automatically discovered through the Windows server address, username and password you provide. The properties and relationships will be periodically updated by modeling.

Device
File systems
Interfaces
Server (Device)
Attributes: Name, Contact, Description, Serial Number, Tag, Hardware Model, Physical Memory, Total Virtual Memory, Operating System, Cluster
Cluster (Device)
Attributes: Name, Contact, Description, Physical Memory, Total Virtual Memory, Operating System, Member Servers
Processors
Attributes: Name, Description, Model, Socket, Cores, Threads, Clock Speed, External Speed, Voltage, L1 Cache Size, L2 Cache Size and Speed, L3 Cache Size and Speed
File System
Attributes: Mount Point, Status, Storage Device, Type, Block Size, Total Blocks, Total Bytes, Maximum Name Length
Interfaces
Attributes: Name, Description, MAC Address, MTU, Speed, Duplex, Type, Administrative Status, Operational Status, IP Addresses
Network Routes
Attributes: Destination, Next Hop, Interface, Protocol, Type
Process Sets
Attributes: Name, Recent Matches, Process Class
Software
Attributes: Name, Vendor, Installation Date
Services
Attributes: Name, Display Name, Start Mode, Account
Cluster Services
Attributes: Name, Core Group, Owner Node, State, Description, Priority
Relationships: Cluster Resources
Cluster Resources
Attributes: Name, Owner Node, Description, Owner Group, State
Relationships: Cluster Service
IIS Sites
Attributes: Name, Status, App Pool
SQL Server Instances
Attributes: Name
Relationships: SQL Server Databases
SQL Server Databases
Attributes: Name, Version, Owner, Last Backup, Last Log Backup, Accessible, Collation, Creation Date, Default File Group, Primary File Path
Relationships: SQL Server Instance
SQL Server Backups
Attributes: Name, Device Type, Physical Allocation, Status
Relationships: SQL Server Instance
SQL Server Jobs
Attributes: Name, Job ID, Description, Enabled, Date Created, Username
Relationships
SQL Server Instance


Performance Monitoring

Perfmon counters are collected using the PowerShell Get-Counter Cmdlet within a remove shell (WinRS). The following metrics will be collected every 5 minutes by default. Any other Windows Perfmon counters can also be collected by adding them to the appropriate monitoring template.

Device-level graphs
File systems
Device
  • \Memory\Available bytes
  • \Memory\Committed Bytes
  • \Memory\Pages Input/sec
  • \Memory\Pages Output/sec
  • \Paging File(_Total)\% Usage
  • \Processor(_Total)\% Privileged Time
  • \Processor(_Total)\% Processor Time
  • \Processor(_Total)\% User Time
  • \System\System Up Time
Active Directory
  • \NTDS\DS Client Binds/sec
  • \NTDS\DS Directory Reads/sec
  • \NTDS\DS Directory Searches/sec
  • \NTDS\DS Directory Writes/sec
  • \NTDS\DS Monitor List Size
  • \NTDS\DS Name Cache hit rate
  • \NTDS\DS Notify Queue Size
  • \NTDS\DS Search sub-operations/sec
  • \NTDS\DS Server Binds/sec
  • \NTDS\DS Server Name Translations/sec
  • \NTDS\DS Threads in Use
  • \NTDS\KDC AS Requests
  • \NTDS\KDC TGS Requests
  • \NTDS\Kerberos Authentications
  • \NTDS\LDAP Active Threads
  • \NTDS\LDAP Bind Time
  • \NTDS\LDAP Client Sessions
  • \NTDS\LDAP Closed Connections/sec
  • \NTDS\LDAP New Connections/sec
  • \NTDS\LDAP New SSL Connections/sec
  • \NTDS\LDAP Searches/sec
  • \NTDS\LDAP Successful Binds/sec
  • \NTDS\LDAP UDP operations/sec
  • \NTDS\LDAP Writes/sec
  • \NTDS\NTLM Authentications
  • \NTDS\DS Client Binds/sec
  • \NTDS\DS Directory Reads/sec
  • \NTDS\DS Directory Searches/sec
  • \NTDS\DS Directory Writes/sec
  • \NTDS\DS Monitor List Size
  • \NTDS\DS Name Cache hit rate
  • \NTDS\DS Notify Queue Size
  • \NTDS\DS Search sub-operations/sec
  • \NTDS\DS Server Binds/sec
  • \NTDS\DS Server Name Translations/sec
  • \NTDS\DS Threads in Use
  • \NTDS\LDAP Active Threads
  • \NTDS\LDAP Bind Time
  • \NTDS\LDAP Client Sessions
  • \NTDS\LDAP Closed Connections/sec
  • \NTDS\LDAP New Connections/sec
  • \NTDS\LDAP New SSL Connections/sec
  • \NTDS\LDAP Searches/sec
  • \NTDS\LDAP Successful Binds/sec
  • \NTDS\LDAP UDP operations/sec
  • \NTDS\LDAP Writes/sec
  • \DirectoryServices(NTDS)\DS Client Binds/sec
  • \DirectoryServices(NTDS)\DS Directory Reads/sec
  • \DirectoryServices(NTDS)\DS Directory Searches/sec
  • \DirectoryServices(NTDS)\DS Directory Writes/sec
  • \DirectoryServices(NTDS)\DS Monitor List Size
  • \DirectoryServices(NTDS)\DS Name Cache hit rate
  • \DirectoryServices(NTDS)\DS Notify Queue Size
  • \DirectoryServices(NTDS)\DS Search sub-operations/sec
  • \DirectoryServices(NTDS)\DS Server Binds/sec
  • \DirectoryServices(NTDS)\DS Server Name Translations/sec
  • \DirectoryServices(NTDS)\DS Threads in Use
  • \DirectoryServices(NTDS)\LDAP Active Threads
  • \DirectoryServices(NTDS)\LDAP Bind Time
  • \DirectoryServices(NTDS)\LDAP Client Sessions
  • \DirectoryServices(NTDS)\LDAP Closed Connections/sec
  • \DirectoryServices(NTDS)\LDAP New Connections/sec
  • \DirectoryServices(NTDS)\LDAP New SSL Connections/sec
  • \DirectoryServices(NTDS)\LDAP Searches/sec
  • \DirectoryServices(NTDS)\LDAP Successful Binds/sec
  • \DirectoryServices(NTDS)\LDAP UDP operations/sec
  • \DirectoryServices(NTDS)\LDAP Writes/sec
Exchange 2003
  • \MSExchangeIS Mailbox(_Total)\Folder opens/sec
  • \MSExchangeIS Mailbox(_Total)\Local delivery rate
  • \MSExchangeIS Mailbox(_Total)\Message Opens/sec
  • \MSExchangeIS\RPC Averaged Latency
  • \MSExchangeIS\RPC Operations/sec
  • \MSExchangeIS\RPC Requests
  • \SMTP Server(_Total)\Local Queue Length
  • \SMTP Server(_Total)\Messages Delivered/sec
Exchange 2007 & 2010
  • \MSExchangeIS Mailbox(_Total)\Folder opens/sec
  • \MSExchangeIS Mailbox(_Total)\Local delivery rate
  • \MSExchangeIS Mailbox(_Total)\Message Opens/sec
  • \MSExchangeIS\RPC Averaged Latency
  • \MSExchangeIS\RPC Operations/sec
  • \MSExchangeIS\RPC Requests
  • \MSExchangeTransport Queues(_Total)\Active Mailbox Delivery Queue Length
  • \MSExchangeTransport SmtpSend(_Total)\Messages Sent/sec
Exchange 2013
  • \MSExchangeIS Store(_Total)\Folders opened/sec
  • \MSExchangeIS Store(_Total)\Messages Delivered/sec
  • \MSExchangeIS Store(_Total)\Messages opened/sec
  • \MSExchange Store Interface(_Total)\RPC Latency average (msec)
  • \MSExchange Store Interface(_Total)\RPC Requests sent/sec
  • \MSExchange Store Interface(_Total)\RPC Requests sent
  • \MSExchangeTransport Queues(_Total)\Active Mailbox Delivery Queue Length
  • \MSExchange Delivery SmtpSend(_Total)\Messages Sent/sec
IIS
  • \Web Service(_Total)\Bytes Received/sec
  • \Web Service(_Total)\Bytes Sent/sec
  • \Web Service(_Total)\CGI Requests/sec
  • \Web Service(_Total)\Connection Attempts/sec
  • \Web Service(_Total)\Copy Requests/sec
  • \Web Service(_Total)\Delete Requests/sec
  • \Web Service(_Total)\Files Received/sec
  • \Web Service(_Total)\Files Sent/sec
  • \Web Service(_Total)\Get Requests/sec
  • \Web Service(_Total)\Head Requests/sec
  • \Web Service(_Total)\ISAPI Extension Requests/sec
  • \Web Service(_Total)\Lock Requests/sec
  • \Web Service(_Total)\Mkcol Requests/sec
  • \Web Service(_Total)\Move Requests/sec
  • \Web Service(_Total)\Options Requests/sec
  • \Web Service(_Total)\Other Request Methods/sec
  • \Web Service(_Total)\Post Requests/sec
  • \Web Service(_Total)\Propfind Requests/sec
  • \Web Service(_Total)\Proppatch Requests/sec
  • \Web Service(_Total)\Put Requests/sec
  • \Web Service(_Total)\Search Requests/sec
  • \Web Service(_Total)\Trace Requests/sec
  • \Web Service(_Total)\Unlock Requests/sec
SQLServer
  • \SQLServer:Access Methods\Full Scans/sec
  • \SQLServer:Buffer Manager\Buffer cache hit ratio
  • \SQLServer:Buffer Manager\Free pages
  • \SQLServer:Databases(_Total)\Data File(s) Size (KB)
  • \SQLServer:General Statistics\User Connections
  • \SQLServer:Latches\Latch Waits/sec
  • \SQLServer:Locks(_Total)\Average Wait Time (ms)
  • \SQLServer:Locks(_Total)\Lock Requests/sec
  • \SQLServer:Locks(_Total)\Number of Deadlocks/sec
  • \SQLServer:SQL Statistics\Batch Requests/sec
File Systems
  • \Disk Read Bytes/sec
  • \% Disk Read Time
  • \Disk Write Bytes/sec
  • \% Disk Write Time
  • \Free Megabytes
Interfaces
  • \Bytes Received/sec
  • \Bytes Sent/sec
  • \Packets Received Errors
  • \Packets Received/sec
  • \Packets Outbound Errors
  • \Packets Sent/sec
IIS Sites
  • \Bytes Received/sec
  • \Bytes Sent/sec
  • \CGI Requests/sec
  • \Connection Attempts/sec
  • \Copy Requests/sec
  • \Connection Attempts/sec
  • \Delete Requests/sec
  • \Files Received/sec
  • \Files Sent/sec
  • \Get Requests/sec
  • \Head Requests/sec
  • \ISAPI Extension Requests/sec
  • \Lock Requests/sec
  • \Mkcol Requests/sec
  • \Move Requests/sec
  • \Options Requests/sec
  • \Other Request Methods/sec
  • \Post Requests/sec
  • \Propfind Requests/sec
  • \Proppatch Requests/sec
  • \Put Requests/sec
  • \Search Requests/sec
  • \Trace Requests/sec
  • \Unlock Requests/sec

The following metrics are collected directly via WMI.

Processes (Win32_PerfFormattedData_PerfProc_Process)
  • PercentProcessorTime
  • WorkingSet
  • WorkingSetPrivate (not available on Windows 2003)


Bulbgraph.png Note: IIS 6 Management compatibility role no longer needs to be installed on the server side in order to use the IIS Sites component. Bulbgraph.png Note: IIS Management Scripts and Tools role needs to be installed on the server side in order to use the IIS Sites component.

Event Management

Events could be collected from the Windows event log using a WinRM subscription. Events collected through this mechanism will be timestamped based on the time they occurred within the Windows event log. Not by the time at which they were collected.

To monitor EventLog events you should add to monitoring template with "Windows EventLog" datasource. For the Event Log field put the name of event log (e.g. "System") that you are interested in, and in the EventQuery you could put the filter for events. Filter is written as PowerShell scriptblock for Where-Object commandlet.

To target all events with a Warning or higher severity:

For Windows 2003: { $$_.EntryType -le [System.Diagnostics.EventLogEntryType]::Warning}

$$_ is the event object of EventLogEntry class. EntryType - is the attribute which determines severity, and could contain one of the following values: Error, Warning, Information, SuccessAudit, or FailureAudit. Also it has such attributes as Message, MachineName, TimeGenerated, Source. Full list you could find at http://msdn.microsoft.com/en-us/library/vstudio/system.diagnostics.eventlogentry .

Note: This query is structured to look for "less than," although we are looking for events "greater than" in severity. This is because the EntryType is an enumeration where the integer values map to 1= Error, 2 = Warning, etc. This means lower numbers indicate higher severity.

For Windows 2008 & Later:

{ $$_.Level -le [System.Diagnostics.Eventing.Reader.StandardEventLevel]::Warning}

The full list of event levels can be found at http://msdn.microsoft.com/en-us/library/system.diagnostics.eventing.reader.standardeventlevel%28v=vs.110%29.aspx

And to know more about writing PowerShell conditions, you could read http://www.powershellpro.com/powershell-tutorial-introduction/powershell-tutorial-conditional-logic/


To change event severity follow the steps:

  1. go to IIS Site events, and click on '/Status' event class.
  2. find 'EventClass Mappings' section and click on 'IISSiteStatus' link
  3. click the Edit button
  4. in the 'Transform' section, add "evt.severity = <severity number(see link http://community.zenoss.org/docs/DOC-3923 in the '#7.1.1.3. severity Field' section)>" at the bottom
  5. click the Save button

Requirements

This ZenPack has the following requirements.

PythonCollector ZenPack
This ZenPack depends on PythonCollector being installed, and having the associated zenpython collector process running.
System Kerberos RPM
The operating system's kerberos RPM must be installed. See the #Installing_Kerberos_Dependency section for details.

Installing Kerberos Dependency

To use kerberos authentication the operating system's kerberos package must be installed on all Zenoss servers. On Enterprise Linux (Red Hat and CentOS) this is the krb5-workstation RPM and can typically be installed by running the following command as the root user.

yum -y install krb5-workstation

Usage

Adding a Windows Device

Use the following steps to start monitoring a Windows server using local authentication in the Zenoss web interface.

  1. Navigate to the Infrastructure page.
  2. Select the Server/Microsoft/Windows device class.
    • The Windows server must be added to this class or to a child of this class.
  3. Click Details and set the configuration properties for zWinRMUser and zWinRMPassword.
  4. Click See All.
  5. Choose Add Single Device from the add device button.
  6. Fill out the form.
    • Name or IP must be resolvable and accessible from the collector server chosen in the Collector field.
  7. Click ADD.

Alternatively you can use zenbatchload to add Windows servers from the command line. To do this, you must create a text file with hostname, username and password of all the servers you want to add. Multiple endpoints can be added under the same /Devices/Server/Microsoft/Windows section. Here is an example...

/Devices/Server/Microsoft/Windows
win2003-1d.example.com zWinRMUser="Administrator", zWinRMPassword="password"
win2008-1d.example.com zWinRMUser="Administrator", zWinRMPassword="password"
Win2012-1d.example.com zWinRMUser="Administrator", zWinRMPassword="password"

You can then load the Windows servers into Zenoss Core or Resource Manager as devices with the following command.

zenbatchload <filename>

Configuration Options

The #Adding a Windows Device steps shown above are for the simplest case of using Windows local authentication. The following configuration properties can be used to support monitoring other environments.

zWinRMUser
The syntax used for zWinRMUser controls whether Zenoss will attempt Windows local authentication or domain (kerberos) authentication. If the value of zWinRMUser is username, local Windows authentication will be used. If zWinRMUser is username@example.com, domain authentication will be used. The zWinKDC and potentially the zWinRMServerName properties become important.
zWinRMPassword
Password for user defined by zWinRMUser.
zWinKDC
The zWinKDC property must be set if domain authentication is used. It must be the IP address of a valid Windows domain controller.
zWinRMServerName
This property should only be used in conjunction with domain authentication when the DNS PTR record for a monitored server's managed IP address does not resolve to the name by which the server is known in Active Directory. For example, if myserver1 is known as myserver1.ad.example.com by Active Directory and is being managed by IP address 192.51.100.21, but 192.51.100.21 resolves to www.example.com, you will have to set zWinRMServerName to myserver1.ad.example.com for domain authentication to work.
If many Windows servers in your environment don't have DNS PTR records that match Active Directory, it is recommended that you set the name of the Zenoss device's to be the fully-qualified Active Directory name and set zWinRMServerName to ${here/titleOrId} at the /Server/Microsoft/Windows device class. This avoids the necessity of setting zWinRMServerName on every device.
It is recommended to leave zWinRMServerName blank if local authentication is used, or DNS PTR records match Active Directory. This allows Zenoss to not rely on DNS resolution while monitoring, and avoids the overhead of configuring zWinRMServerName.
zWinScheme
This must be set to either http or https. The default is http.
zWinRMPort
The port on which the Windows server is listening for WinRM or WS-Management connections. The default is 5985. It is uncommon for this to be configured as anything else.
zWinPerfmonInterval
The default interval in seconds at which Windows Perfmon datapoints will be collected. The default is 300 seconds or 5 minutes. It is also possible to override the collection interval for individual counters.
zWinKeyTabFilePath
This property is currently used and reserved for future use when keytab files are supported.
zDBInstances
This setting is only relevant when the zenoss.winrm.WinMSSQL modeler plugin is enabled. Multiple instances can be specified to monitor multiple SQL Server instances per server. The default instance is MSSQLSERVER. Fill in the user and password to use SQL authentication. Leave the user and password blank to use Windows authentication.

Configuring Service Monitoring

There are multiple ways to configure Windows service monitoring depending on if you want to configure for a single service on a single server, a specific service across all Windows servers, or somewhere in between. The following flowchart describes how Zenoss decides whether a service will be monitored, and subsequently what monitoring template will be used to monitor the service.

Windows Service Monitoring Flowchart

This allows for the following example uses.

Enable or disable monitoring for a single service on a single server.
  1. Navigate to the service on the server.
  2. Click to select it.
  3. Choose Monitoring from the gear menu.
  4. Choose Yes or No depending on what you want.
Enable monitoring by default for the WinRM service wherever it is enabled.
  1. Navigate to Advanced -> Monitoring Templates.
  2. Verify the list of templates is grouped by template.
  3. Expand the WinService tree.
  4. Click once to select the /Server/Microsoft copy.
  5. Choose Copy / Override Template from the gear menu.
  6. Select /Server/Microsoft (Create Copy) from the target list then click submit.
  7. Expand the resulting copy_of_WinService tree.
  8. Select the /Server/Microsoft copy.
  9. Choose View and Edit Details from the gear menu.
  10. Change the template's name to WinRM then click submit.
  11. Double-click to edit the DefaultService' datasource.
  12. Put a check mark in Monitor by Default then click save.
Enable monitoring by default for the WinRM service for a select group of servers.
  1. Create a new device class somewhere under /Server/Microsoft/Windows for the select group of servers.
  2. Move the servers to the new device class.
  3. Follow steps 1-5 from the previous section.
  4. Choose your new device class as the target then click submit.
  5. Expand the WinService tree then select the copy in your device class.
  6. Choose View and Edit Details from the gear menu.
  7. Change the template's name to WinRM then click submit.
  8. Double-click to edit the DefaultService' datasource.
  9. Put a check mark in Monitor by Default then click save.


Configuring MSSQL Server Modeling/Monitoring

Supported SQL Server versions
SQL Server 2005
SQL Server 2008
SQL Server 2008 R2
SQL Server 2012
Support for SQL Server and Windows Authentication
  • Windows Authentication: In zDBInstances property specify only SQL instances names, leave user and password fields blank.
  • SQL Server Authentication: In zDBInstances property provide user name and password for each SQL instance.

Use the following steps to configure SQL Server Authentication on your SQL Server:

  1. Connect to SQL Instance using MSSQL Management Studio.
  2. Select instance Properties > Security and make sure that SQL Server and Windows Authentication mode is enabled.
  3. Open Security > Logins, select the user, you specified in zDBInstances property.
  4. Check user Properties > Status and make sure that the user is Enabled.
  5. Check user Properties > Server Roles and make sure that the user has sysadmin and public roles.
Support for Local and Failover Cluster SQL instances

This ZenPack adds support for both local and failover cluster SQL Server instances. Local SQL Server instances can be modeled/monitored within windows devices (devices in Server/Microsoft/Windows device class). SQL Server failover cluster instances can be modeled/monitored within cluster devices (devices in Server/Microsoft/Cluster device class).

Use the following steps to model/monitor SQL Server instances:
  1. Create a device in Server/Microsoft/Windows device class if you intend to model local SQL instances, or in Server/Microsoft/Cluster device class if you intend to model failover cluster instances.
  2. Specify the instance names to be modeled in zDBInstances zProperty. Provide user names and passwords if SQL Server Authentication is to be used.
  3. Enable zenoss.winrm.WinMSSQL modeler plugin.
  4. Remodel device.

Working with WinCommand Notification Action

This ZenPack adds a new event notification action that can be used by the zenactiond daemon to allow an arbitrary command to be executed on the remote windows machine.

Use the following steps to set up a notification:

  1. Select Events > Triggers from the Navigation Menu.
  2. Create a trigger, selecting the rules that define it.
  3. Select Notifications from the left panel. Add a new notification, enter a name for it and select WinCommand Action from the drop-down menu. Click Submit.
  4. In the Edit Notification dialog on the Notification tab associate the trigger with the notification and optionally select the notification properties (Enabled, Send Clear, Send only on Initial Occurrence, Delay, Repeat).

On the Content tab of the notification specify the 'Windows CMD Command to run when configured triggers are matched. You may optionally specify Clear Windows CMD Command to run when the triggering event clears.

  1. Submit changes.

For more information please refer to Working with Triggers and Notifications

Setting up WinRM Service for Target Windows Machines

Windows server operating systems from Windows 2003 SP1 are supported. The reason for this is that WinRM version 2 is required, and it was only added in Windows 2003 SP1.

Group Policy

Computer Configuration\Policies\Administrative Templates\Windows Components\Windows Remote Management

WinRMClient

  • No setting changes required for client

WinRMService

  • Allow remote server management through WinRm

- HTTP (Windows default is HTTPS see note below for more information)

  • Allow unencrypted Traffic

- Basic Authentication (Windows default is Kerberos see note below for more information)

  • Allow Basic Authentication

WinRS Computer Configuration\Policies\Administrative Templates\Windows Components\Windows Remote Shell

  • Allow Remote Shell Access
  • Max number of processes per shell = 4294967295
  • Max number of shells per user = 2147483647
  • Shell Timeout = 7200000

Individual Machine configuration

  • Open ports 5985 (http)/5986(https) for WinRM
  • Run command prompt as Administrator
  • winrm quickconfig
  • winrm s winrm/config/service @{MaxConcurrentOperationsPerUser="4294967295"}
  • winrm s winrm/config/winrs @{MaxShellsPerUser="2147483647"}
  • winrm s winrm/config/winrs @{IdleTimeout="7200000"}

Basic Authentication (Windows default is Kerberos see note below for more information)

  • winrm s winrm/config/service/auth @{Basic="true"}
  • winrm s winrm/config/service @{AllowUnencrypted="true"}

Bulbgraph.png Note: The above instructions use the max values for MaxConcurrentOperationsPerUser and WinRS MaxShellsPerUser. If you do not want to set these to the max, then a value of 50 should be adequate. The default is 5 on both, which will cause problems because Zenoss will open up concurrent requests for each WQL query and set of Perfmon counters.

Bulbgraph.png Note: If you choose to use Basic authentication it is highly recommended that you also configure HTTPS. If you do not use the HTTPS protocol your user name and password will be sent over in clear text. If you have challenges setting up HTTPS on the Windows clients but require the user name and password to be encrypted, then using the Kerberos authentication is the best option. HTTPS is not required for Kerberos but is recommended. If you choose to use Kerberos authentication, then your payload will be encrypted.

Bulbgraph.png Note: If you choose to take the WinRM default configurations you must supply Kerberos authentication settings in the zProperties. The Kerberos authentication process requires a ticket granting server. In the Microsoft Active Directory environment the AD Server is also the KDC. The zWinKDC value must be set to the IP address of the AD Server and the collector must be able to sent TCP/IP packets to this server. Once this is set your zWinRMUserName must be a FQDN such as jsmith@Zenoss.com and the zWinRMPassword must be set correctly for this user account.

Bulbgraph.png Note: The HTTPS setup must be completed on each client. At this time we do not have notes on automating this task but are currently in the process of testing several options. To successfully encrypt your payload between the Zenoss server and the Windows client you must install a Server Authentication certificate on the client machine. The process for requesting and installing the appropriate certificate can be found at the following URL. http://blogs.technet.com/b/meamcs/archive/2012/02/25/how-to-force-winrm-to-listen-interfaces-over-https.aspx Once the client has the correct certificate installed you only need to change the zWinScheme to HTTPS and zWinRMPort to 5986. If you are still having challenges setting up HTTPS on the client you can execute the following command on any AD server to verify the appropriate SPN record exists for Kerberos authentication.

setspn -l hostname1

If you do not see a record with HTTPS/ at the beginning of the hostname you will need to create the record.

example: setspn -s HTTPS/hostname1.zenoss.com hostname1


Transitioning from WindowsMonitor

If you are installing this ZenPack on an existing Zenoss system or upgrading from an earlier Zenoss version you may have a ZenPack named ZenPacks.zenoss.WindowsMonitor already installed on your system. You can check this by navigating to Advanced -> ZenPacks.

This ZenPack functionally supersedes ZenPacks.zenoss.WindowsMonitor for Windows platforms that support WinRM, but does not automatically migrate monitoring of your Microsoft Windows resources when installed. The ZenPacks can coexist gracefully to allow you time to manually transition monitoring to the newer ZenPack with better capabilities.

  1. Navigate to the Infrastructure page.
  2. Expand the Server/Windows/WMI device class.
  3. Single-click to select a Windows device.
  4. Click the delete (-) button in the bottom-left.
  5. Click OK to confirm deleting the Windows device.
  6. Add the device back using the #Adding a Windows Device instructions above. Be sure to select the /Server/Microsoft/Windows device class and not the /Server/Windows/WMI device class.
  7. Repeat steps 3-6 for each Windows device.

Old Windows ZenPacks:

  • PySamba
  • WindowsMonitor
  • ActiveDirectory
  • IISMonitor
  • MSExchange
  • MSMQMonitor
  • MSSQLServer

New Windows ZenPacks:

  • PythonCollector is a requirement for this ZenPack. It provides the polling facility through the zenpython collector daemon.
  • This ZenPack (all the functionality of the old Windows ZenPacks is rolled into this one ZenPack)

The old ZenPacks come as part of the Zenoss Core 4.2.x RPM. They can be installed on top of an RM install with the msmonitor RPM

Using Old and New Windows ZenPacks Together

There are some scenarios where it may be useful to use the old and new Windows ZenPacks together. In most cases this is as simple as putting servers you wish to be monitored by the new ZenPack in the /Server/Microsoft/Server device class and servers you wish to be monitored by the old ZenPack in the /Server/Windows/WMI device class.

Due to this ZenPack's dependency on WinRM 2.0 it is not possible to monitor Windows versions earlier than Windows 2003 SP1. If you have a requirement to monitor these earlier Windows versions you must use the older WindowsMonitor ZenPack that uses DCOM/RPC instead of WinRM.

There are also circumstances where you may currently be using the old Windows ZenPack and only want to initially use the new ZenPack for its new functionalities such as the Windows Shell datasource. This can be achieved using the following steps.

  1. Keep the servers under the /Server/Windows/WMI device class.
  2. Verify that all of the following configuration properties are set:
    • zWinUser: In DOMAIN\Username format for DCOM/RPC collection.
    • zWinPassword: Password for zWinUser account.
    • zWinRMUser: In username@example.com format for WinRM collection.
    • zWinRMPassword: Password for zWinRMUser account.
  3. Create a monitoring template containing a Windows Shell datasource and bind it to the server.

Limitations of Current Release

The current release is known to have the following limitations.

  • Support for team NICs is limited to Intel and Broadcom interfaces.
  • The custom widget for MSSQL Server credentials is not compatible with Zenoss 4.1.x, therefore the zDBInstances property in this version should be set as a valid JSON list (e.g. [{"instance": "MSSQLSERVER", "user": "", "passwd": ""}] ).
  • When upgrading to version 2.2.0, you may see a segmentation fault during the install. This occurs when upgrading from versions 2.1.3 and previous. To ensure a successful installation, run the install once more and restart Zenoss.
  • Payload encryption is not currently supported for version 2.2.0. Please use basic authentication.

A current list of known issues related to this ZenPack can be found with this JIRA query. You must be logged into JIRA to run this query. If you don't already have a JIRA account, you can create one here.

Manually Establishing Kerberos Tickets

In version 2.0.0 of the ZenPack there is a problem in the automatic establishment of kerberos tickets required for monitoring Windows devices using kerberos authentication. This is only a problem on Enterprise Linux 5 (Red Hat or CentOS). It is not a problem on Enterprise Linux 6. The problem will typically manifest as the following error when attempting to model a Windows device.

Bulbgraph.png Note: Note that these manual steps are not necessary in version 2.0.1 and later of the ZenPack.

kerberos authGSSClientStep failed (None)

It is possible to workaround this error by manually establishing the tokens using the following steps.

  1. Edit $ZENHOME/var/krb5/krb5.conf.
    1. Remove the includedir line.
    2. Add the following to the bottom of the file.
[realms]
 EXAMPLE1.COM = {
  kdc = 192.168.77.77 #KDC IP Address or FQDN
  admin_server = 192.168.77.77 #KDC IP Address or FQDN
 }
 EXAMPLE2.COM = {
  kdc = 192.168.88.88 #KDC IP Address or FQDN
  admin_server = 192.168.88.88 #KDC IP Address or FQDN
 }

[domain_realm]
 .example1.com = EXAMPLE1.COM
 example1.com = EXAMPLE1.COM
 .example2.com = EXAMPLE2.COM
 example2.com = EXAMPLE2.COM

This is an example of what would be required if you had two domains: example1.com and example2.com with domain controllers at 192.168.77.77 and 192.168.88.88 respectively. You can use a single domain or more than two. Be sure to use the same capitalization scheme.

Service Impact

When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact capability for services running on Microsoft Windows. The following service impact relationships are automatically added. These will be included in any services that contain one or more of the explicitly mentioned entities.

Service Impact Relationships
  • The Windows server is impacted by the Processors and File Systems.

Troubleshooting

Please refer the the Zenoss Service Dynamics documentation if you run into any of the following problems:

  • ZenPack will not install
  • Adding a device fails
  • Don't understand how to add a device
  • Don't understand how to model a device

If you cannot find the answer in the documentation, then Resource Manager (Service Dynamics) users should contact Zenoss Customer Support. Core users can use the #zenoss IRC channel or the community.zenoss.org forums (there is a forum specific to Windows monitoring).

Troubleshooting Kerberos Error Messages

Cannot determine realm for numeric host address
  • If you enter an IP address for the device id, make sure that the address is resolvable to a name.
Server not found in Kerberos database
  • Either the HTTP or HTTPS service principal name cannot be found on the device. You can add it using the setspn program on the Windows Server.

example: setspn -s HTTP/hostname1.zenoss.com hostname1

Troubleshooting Kerberos Authentication with Wireshark

There are many reasons for kerberos authentication not to work, and a lot of them result in the following unhelpful error message.

kerberos authGSSClientStep failed (None)

While Zenoss is unable to extract a useful error message when this occurs, it turns out that Wireshark can get useful errors by looking at the kerberos packets sent between Zenoss, your domain controller (zWinKDC) and the monitored Windows server. Let's walk through an example of using Wireshark to resolve an authGSSClientStep failed error.

  1. First install Wireshark on your system. It's GUI is easier to use than the command line equivalent.
  2. Next you will need to create a packet capture file on your Zenoss server. Assuming the Windows server you're trying to monitor is 192.0.2.101 and the domain controller (zWinKDC) is 203.0.113.10, you would run the following command as the root user on your Zenoss server.
    tcpdump -s0 -iany -w kerberdebug.pcap host 192.0.2.101 or host 203.0.113.10
    This will start capturing all packets to or from those two IP addresses. It will continue to capture these packets until you type CTRL-C.
  3. Now you should attempt to remodel the Windows server where you're encountering the error. Once it completes, and fails, again you should go back to the terminal where tcpdump is running and type CTRL-C. You will now have a kerberdebug.pcap file in the directory where you ran the command.
  4. Copy kerberdebug.pcap to your system where you installed Wireshark. Start Wireshark and open kerberdebug.pcap. You should see something like the following.
    Windows-kerberos-wireshark.png

You'll see that there's a KRB5KRB_AP_ERR_SKEW error. Searching for this specific error code will quickly show that it occurs when the kerberos client and server don't have their time's synchronized. There's a tolerance for some difference, but in this case it was a big difference due to misconfiguration.

There are some kerberos errors you'll see in the packets that a completely normal part of negotiation and won't lead to any problems. You should ignore the following errors shown in Wireshark:

  • KRB5KRB_API_ERR_TKT_EXPIRED: Zenoss will subsequently request a new ticket when this occurs.
  • KRB5KRB_ERR_PREAUTH_REQUIRED: This is a normal part of kerberos negotiation.
  • KRB5KRB_ERR_RESPONSE_TOO_BIG: Most requests won't fit in UDP. Zenoss will automatically switch to TCP.

You'll also see other kerberos messages that are normal. You should ignore these kerberos messages shown by Wireshark:

  • TGS-REQ
  • AS-REQ

The following are the most common errors:

  • KRB5KRB_AP_ERR_SKEW: As shown in the above example. A clock synchronization issue.
  • KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN: This can happen if zWinRMServerName resolves to the server's IP address, but is not the name the server is known by in Active Directory. This will also be the error if you don't enter a zWinRMServerName and the reverse resolution of the device's manage IP address resolves to a name that doesn't match the server's name in Active Directory. Typical solutions to this are to add the name to the /etc/hosts file or to directly use the IP address of the server.

Troubleshooting on Resource Manager 4.1.1

In some cases updating the Microsoft Windows ZenPack on Zenoss Resource Manager 4.1.1 may result in the zenhub daemon not starting. The error message will contain AttributeError: zDBInstancesPassword. If you encounter this issue, install the ZenPack again.

If there are existing SQL server instances being monitored, make sure to reconfigure zDBInstances property since the zDBInstancesPassword property no longer exists.

Troubleshooting Services

If you see an event error that shows "The maximum number of concurrent operations for this user has been exceeded", you will need to increase the number of concurrent operations per user in the winrm config. For example:

  • winrm set winrm/config/service '@{MaxConcurrentOperationsPerUser="5000"}'

If you see an "Index out of range" error, this could indicate a low number of available file handles in Linux. The default is 1024. To view this information on your system, enter 'ulimit -n'. To increase this limit, edit your /etc/sysctl.conf file and set fs.file-max to a sufficently large number. For example:

  • vi /etc/sysctl.conf
  • fs.file-max=10000

Zenoss Analytics

This ZenPack provides additional support for Zenoss Analytics. Perform the following steps to install extra reporting resources into Zenoss Analytics after installing the ZenPack.

  1. Copy analytics-bundle.zip from $ZENHOME/ZenPacks/ZenPacks.zenoss.Microsoft.Windows*/ZenPacks/zenoss/Microsoft/Windows/analytics/ on your Zenoss server.
  2. Navigate to Zenoss Analytics in your browser.
  3. Login as superuser.
  4. Remove any existing Microsoft Windows ZenPack folder.
    1. Choose Repository from the View menu at the top of the page.
    2. Expand Public in the list of folders.
    3. Right-click on Microsoft Windows ZenPack folder and choose Delete.
    4. Confirm deletion by clicking OK.
  5. Add the new Microsoft Windows ZenPack folder.
    1. Choose Server Settings from the Manage menu at the top of the page.
    2. Choose Import in the left page.
    3. Remove checks from all check boxes.
    4. Click Choose File to import a data file.
    5. Choose the analytics-bundle.zip file copied from your Zenoss server.
    6. Click Import.

You can now navigate back to the Microsoft Windows ZenPack folder in the repository to see the following resources added by the bundle.

Domains
  • Microsoft Windows Domain
  • Microsoft Cluster Domain
Ad Hoc Views
  • Windows IIS Peak Usage
  • Windows Interfaces Peak Usage

Domains can be used to create Ad Hoc views using the following steps.

  1. Choose Ad Hoc View from the Create menu.
  2. Click Domains at the top of the data chooser dialog.
  3. Expand Public then Microsoft Windows ZenPack.
  4. Choose the Microsoft Windows Domain domain

Installed Items

Installing this ZenPack will add the following items to your Zenoss system.

Device Classes
  • /Server/Microsoft
  • /Server/Microsoft/Cluster
  • /Server/Microsoft/Windows
Configuration Properties
  • zWinRMUser
  • zWinRMPassword
  • zWinRMServerName
  • zWinRMPort
  • zDBInstances
  • zWinKDC
  • zWinKeyTabFilePath
  • zWinScheme
  • zWinPerfmonInterval
Modeler Plugins
  • zenoss.winrm.CPUs
  • zenoss.winrm.FileSystems
  • zenoss.winrm.IIS
  • zenoss.winrm.Interfaces
  • zenoss.winrm.OperatingSystem
  • zenoss.winrm.Processes
  • zenoss.winrm.Routes
  • zenoss.winrm.Services
  • zenoss.winrm.Software
  • zenoss.winrm.WinCluster
  • zenoss.winrm.WinMSSQL
Datasource Types
  • Windows EventLog
  • Windows IIS Site
  • Windows Perfmon
  • Windows Process
  • Windows Service
  • Windows Shell
Monitoring Templates
  • Device (in /Server/Microsoft)
  • FileSystem (in /Server/Microsoft)
  • ethernetCsmacd (in /Server/Microsoft)
  • OSProcess (in /Server/Microsoft)
  • OSProcess-2003 (in /Server/Microsoft)
  • WinService (in /Server/Microsoft)
  • Active Directory (in /Server/Microsoft)
  • Active Directory 2008 (in /Server/Microsoft)
  • Active Directory 2008R2 (in /Server/Microsoft)
  • IIS (in /Server/Microsoft)
  • IISADMIN (in /Server/Microsoft)
  • IISSites (in /Server/Microsoft)
  • MSExchangeIS (in /Server/Microsoft)
  • MSExchange2010IS (in /Server/Microsoft)
  • MSExchange2013IS (in /Server/Microsoft)
  • MSSQLServer (in /Server/Microsoft)
  • WinDatabase (in /Server/Microsoft)
  • Cluster (in /Server/Microsoft)
  • ClusterService (in /Server/Microsoft/Cluster)
  • ClusterResource (in /Server/Microsoft/Cluster)

Changes

2.2.0
  • Payload encryption over kerberos connections
  • Updated Events to use Get-WinEvent cmdlet
  • Updated Software modeler to query registry instead of Win32_Product
  • Updated FileSystems to show mapped network drives and mounted volumes
  • Support for Zenoss Analytics
  • Numerous bug fixes
2.1.3
  • Zenoss 5 compatibility fixes.
2.1.2
  • Added WinCommand notification action
  • Support for monitoring fail-over clustered MSSQL instances
  • Support for monitoring Windows event logs
  • Numerous bug fixes
2.1.0
  • Support for Service Impact
  • Support for Microsoft Exchange 2010 and Microsoft Exchange 2013
  • Ability to monitor Microsoft SQL Server using Windows Authenticated user
  • Fix Exchange 2007 counters
  • Fix cluster and node relationship
  • Fix virtual network adapter monitoring
2.0.3
  • Reduce possibility of gaps in perfmon collection. (ZEN-10600)
  • Add zWinRMServerName property. (ZEN-9712)
  • Support for IIS 7-8 without IIS 6 compatibility.
  • Honor sequence in process monitoring. (ZEN-10777)
  • Fix cluster modeling for long server names. (ZEN-10572)
  • Support TALES in Windows Shell custom command script. (ZEN-10426)
  • Fix custom parser issue with Windows Shell datasource. (ZEN-10365)
  • Handle null software install date. (ZEN-10361)
  • Handle null process socket designation. (ZEN-10360)
  • Model interface speed as integer. (ZEN-9608)
  • Change WinRS success events from info to clear severity.
  • Fix leaking of active operations on Windows server.
  • Add missing counter details to missing counter events.
  • Fix Windows Shell collection on empty results.
  • Fix Windows Perfmon collection with cycletime > 600.
2.0.2
  • Fix build issue that made ZenPack unavailable from catalog.
2.0.1
  • Eliminate need for manual kerberos configuration on Enterprise Linux 5. (ZEN-9389)
  • Fix "WinServiceLog: failed collection" error. (ZEN-9607)
  • Provide more helpful error if AllowUnencrypted is disabled. (ZEN-9524)
2.0.0
  • Initial release of new Windows support using WinRM instead of DCOM/RPC.

Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.Microsoft.Windows-*.egg
  4. Restart these services:
    $ zenoss restart

Developer Mode Installation

In order to do a development mode installation you will want to clone the existing git repository, and then use the --link flag with the zenpack command:

  1. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  2. Start by cloning the upstream repository:
    $ git clone https://github.com/zenoss/ZenPacks.zenoss.Microsoft.Windows.git
  3. Next, perform the installation:
    $ zenpack --link --install ZenPacks.zenoss.Microsoft.Windows
  4. Finally, restart these serivices:
    $ zenoss restart

Discuss

Purplemarker.png New: Don't forget to add yourself to the Zenoss User Map!

blog comments powered by Disqus