Core QA Test Plan

From Zenoss Wiki
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

This page is intended to define a collaborative QA Test plan for Zenoss Core that will be implemented prior to every Zenoss Core release, both new releases as well as maintenance releases.

What Is Tested Internally

Zenoss, Inc. has a QA process that is used prior to a Core release, but it is focused on our commercial products: Zenoss Resource Manager and Zenoss Service Dynamics. There are important differences between our Zenoss Core and our commercial offerings. Parts of Zenoss are changed in the commercial products by the addition of ZenPacks. Although ZenPacks are commonly understood as a means of adding new functionality to Zenoss, they can also be used to alter core internal functionality. We often use ZenPacks in this way in our commercial product offerings to add certain capabilities required by our enterprise customers.

The internal Zenoss, Inc. QA process tests Zenoss in its commercial form, with our commercial ZenPacks applied. Once the QA process is completed successfully, a build is selected for release. This build is used as the basis for both commercial as well as Core (Open Source) releases. This current methodology does expose Zenoss Core to certain gaps in testing, which this Core QA Test Plan is intended to address.

Zenoss Core and Commercial Differences

The following functional areas in Zenoss Core are augmented, overridden or replaced in some way by Zenoss commercial ZenPacks. These functional areas require additional Zenoss Core testing since they are not tested by our existing internal QA process, which is performed with the commercial ZenPacks applied.

  • Collectors (Daemon Framework & Model/UI)
  • Google Maps
  • Global Search
  • Linux SSH Monitoring
  • ZODB Catalog
  • Audit Logging
  • Password Storage (zProperties)
  • Call Home

QA Test Plan Coordination

The general process for the QA test plan is as follows:

  • Approximately 4 weeks prior to an expected new release, Zenoss Engineering will inform Zenoss Open Source that a release is imminent.
  • Zenoss Open Source will initiate the collaborative Zenoss Core QA Test Plan:
    • Zenoss Engineering will upload beta builds to a public download location
    • Zenoss Open Source will provide an auto-deploy script for testing these builds.
    • Zenoss Open Source will coordinate with key open source users to perform live product testing and evaluation.
    • Bugs will be filed on Zenoss JIRA by open source users.
    • Zenoss Open Source will coordinate with open source users to determine the impact of these bugs on the release, and determine which issues are considered blockers for release.
    • Zenoss Open Source will attend internal defect review meetings and act as advocates for these blocking bugs
    • Zenoss Engineering will make a best effort to ensure that they are fixed prior to the release being frozen.
    • A single build will be chosen for both commercial and Open Source releases once Core and internal QA testing completes successfully.