Upgrading Zenoss

From Zenoss Wiki
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

Upgrading ZenOSS 3.2+ -> ZenOSS 4.1+

Notes:

  • This can be hit and miss depending on the consistency of your ZODB, though, no where near as much as upgrading version 2 to 3.
  • It's important to note that converting the Events database from Zenoss v3.x to the v4.x format is unsupported. It's not impossible, but I'm not going to cover it.
  • Do not skip the backup. If you do, that's on you. You must back it up.
  • Installations covering dedicated mysql database servers and remote collectors is not covered.
  • This doesn't cover stack installations or installations moving from a 32-bit install.
  • Zenfixit.py can be found on the forums. DO NOT use zenfixit.py after upgrading to version 4+

1) Backup the current instance, minimizing the disk space used. You can adjust the tar command exclusions if you'd like to backup more.

On ZenOSS master as zenoss user

$ cd $ZENHOME
$ zeopack.py -h localhost -p 8100

Change user to root

$ cd /opt
$ tar czvf ZenOSS.v3.Backup.{DATE}.tar.gz zenoss/ --exclude=*.rrd --exclude=*.old --exclude=*.bak --exclude=*.example --exclude=*.tar --exclude=*.gz --exclude=*.tgz --exclude=*.zip --exclude=*.log*

where {DATE} is the date in YYYY-MM-DD format. Please put a date, it's especially handy if you frequently backup.

At this point you'll have a nice big tar.gz with a complete backup of your ZenOSS with the notable exclusion of the Events database and RRD files.

2) Determine what packs you're using, and compare these to the ZenPack Catalog, if there is an updated version for 4.x install it, if not move the devices to /Discovered and delete the pack. If you have packs you are not using, move any devices to /Discovered and remove them. If you are in doubt about a zenpack, REMOVE IT, most upgrade failures are related to ZenPacks. Each ZenPack installed raises the possibility of total and unmitigated failure during cataloging and zenmigration. Seriously, that ZenPack you decided to keep, remove it now. Zenpacks should be removed as the zenoss user on the ZenOSS master via the console. Please do not remove zenpacks via the webUI as trace back data may not be visable and pack uninstallation failures will need to be addressed before upgrading.

On the ZenOSS master as the zenoss user

$ zenpack --remove ZenPacks.Node2.Node3

3) Cleanup Zope. Zope can become full of trash and the degree of trash can vary based on the Administrator, packs utilized, dimensions of the instance, development of packs on production devices, upgrade history, etc.

The quickest way to clean up Zope is to run `zenfixit.py`. ZenFixIt has sort of a checked history. It can indeed fix many things, however, it can unfix certain items as well. In most cases ZenFixIt will not cause damage but infact fix problems which raise new problems. All of these issues need to be dealt with before upgrading to v4.x+.

# wget http://community.zenoss.org/servlet/JiveServlet/download/2003-13132-47157-2585/zenfixit.py.txt.zip
# unzip zenfixit.py.txt.zip
# mv zenfixit.py.txt zenfixit.py
# python zenfixit.py

4) Now that our Zope is squeaky clean and we've removed all unnecessary ZenPacks we can proceed to shut down Zenoss and MySQL, and then uninstall and reinstall MySQL. As the root user:

# service zenoss stop
# service mysqld stop

Note: you need to use 'su -' not just 'su' in order for the service command to be available.

Then list the installed mySQL RPM's:

# rpm -qa | grep mysql
mysql-5.0.95-5.el5_9
mysql-server-5.0.95-5.el5_9
mysql-5.0.95-5.el5_9

Uninstall the installed MySQL RPM's (adjust the names to match the modules listed in the command output above):

# rpm -e mysql-server-5.0.95-5.el5_9 --nodeps
# rpm -e --allmatches mysql-5.0.95-5.el5_9 --nodeps

Delete the old MySQL /var/lib/mysql (since the events won't be migrated anyway and a clean MySQL database makes things easier):

# rm -rf /var/lib/mysql

Core - Download and install MySQL

# wget http://wiki.zenoss.org/download/core/mysql/MySQL-client-5.5.28-1.linux2.6.x86_64.rpm
# wget http://wiki.zenoss.org/download/core/mysql/MySQL-server-5.5.28-1.linux2.6.x86_64.rpm
# wget http://wiki.zenoss.org/download/core/mysql/MySQL-shared-5.5.28-1.linux2.6.x86_64.rpm
# wget http://wiki.zenoss.org/download/core/mysql/MySQL-shared-compat-5.5.28-1.linux2.6.x86_64.rpm
# yum localinstall MySQL-* --nogpg

Commercial/Enterprise - Please install zends 5.5.25a or the latest supported recommended version.

Increase the memory allocation to MySQL, otherwise data migration can fail part way through:

# vi /etc/my.cnf

Add the lines:

[mysqld]
max_allowed_packet=16M
innodb_buffer_pool_size=256M
innodb_additional_mem_pool_size=20M

5) Download and install Java

   Remove OpenJDK. OpenJDK is not supported.
   Download JRE or JDK 6. Java 7 is not supported. 
   Install Java and verify using java -version
   Set $JAVA_HOME and add the new Java to alternatives.
   Verify with `java -version | grep 'HostSpot'

6) Now we can install all remaining dependencies. First, add the Zenoss Dependencies Repository

If you have RHEL/CentOS 5:

# rpm -Uvh http://deps.zenoss.com/yum/zenossdeps-4.2.x-1.el5.noarch.rpm
# yum clean all

If you have RHEL/CentOS 6:

# rpm -Uvh http://deps.zenoss.com/yum/zenossdeps-4.2.x-1.el6.noarch.rpm
# yum clean all

Install All Dependencies

# yum install memcached
# yum install rabbitmq-server
# yum install nagios-plugins nagios-plugins-dig nagios-plugins-dns nagios-plugins-http nagios-plugins-ircd nagios-plugins-ldap nagios-plugins-ntp nagios-plugins-perl nagios-plugins-ping nagios-plugins-rpc nagios-plugins-tcp redis rrdtool

Start it all up and Turn it on

# service memcached start
# service rabbitmq-server start
# service {mysql or zends} start
# chkconfig {mysql or zends} on
# chkconfig zenoss on
# chkconfig memcached on
# chkconfig rabbitmq-server on

7) Now it's time to install ZenOSS. Before we install there's a number of potential conflicts aloof that we should go ahead and work around right now. On the ZenOSS master as the zenoss user:

# cd $ZENHOME
# mv lib lib.V3
# mv include include.V3
# mv Products Products.V3
# mv share share.V3

These are all folders that can potentially cause code conflicts. Once complete, as the root user on the ZenOSS master:

# rpm -Uvh {zenoss4.Xcore}.rpm

where {zenoss4.Xcore} is the name of the 4.X RPM

8) Now we're ready to start up the freshly upgraded ZenOSS. As the root user on the ZenOSS master

# service zenoss start

During the first `zenoss start` post upgrade the ZenOSS startup script runs the $ZENHOME/bin/zenoss_pre_upgrade and $ZENHOME/bin/zenoss_post_upgrade scripts.

$ZENHOME/bin/zenoss_pre_upgrade confirms and upgrades a number of files, migrates the ZODB and transaction history to relStorage and runs zenmigrate. Watch these steps as they progress.

*** If the parsing of the Zope.conf, migration of the ZODB or Zenmigrate returns any traceback or appears to have failed, you'll need to address the problem and then stop and start ZenOSS.
* Do not use the restart option of the ZenOSS startup script as it does not run the preliminary scripts.
* This can take substantial amounts of time for jumbo installations but is relatively fast for small and medium instances. 
* If you modified the snmp_linkup or snmp_linkdown transform code, the zenmigrate may fail when it tries to edit the transform. To correct this, edit the /opt/zenoss/Products/ZenModel/migrate/fixLinkUpDownTransforms.py file and comment out the last line: FixLinkUpDownTransforms()   by changing it to #FixLinkUpDownTransforms()
* If efforts to fix the issues cannot be bypassed, you may need to consider restoring the backup and removing all ZenPacks before retrying. (Recursive ZenPack removal)

9) At this point we can go ahead and upgrade our ZenPacks. On the ZenOSS master as the zenoss user

$ cd $ZENHOME/packs
$ zenpack --install {filename}

where the filename represents the packs you currently have installed in ZenOSS. You can get this list via

$ zenpack --list

You must upgrade all your currently installed packs to the newer versions.

10) Almost there! After restarting ZenOSS on the ZenOSS master a final time, check the webUI and functionality. If everything appears in order push the new configuration to your collectors on the collectors page and consider deleting all .V3 folders from earlier.