Broadcom / LSI MegaRAID RAID Cheat Sheet

Background

When you’re running your own hardware you need to carefully consider disk redundancy. Apart from healthy current backups, disk redundancy is one of the most important things you’ll ever do when running a server setup.

This articles introduces some concepts around RAID, which is what people use for disk redundancy. It also has specific focus on the Broadcom LSI MegaRAID controllers that seems to be universal in data centres for owned / bare metal hardware. Finally the article moves on to help you actively monitor RAID. It doesn’t help you have RAID set up but in the data centre your disks are failing and you’re blissfully unaware. When RAID fails the busy administrator needs to be be acutely aware that they must go and change the disk. This will require a lot of insight into where is what and what is the status.

Why did we make this article? The reason is there are many concepts and finding good and pertinent information on Google is a mess. Not only does Broadcom have poor documentation, but other articles on the internet is totally overwhelming with technical jargon and lingo whereas the busy administrator just wants to skip to “getting it to work”.

RAID Levels

Let’s start with an important point here, namely a quick summary of RAID levels:

RAID LevelNumber of DisksUseComment
RAID 02Performance (striping)No redundancy
RAID 12RedundancyMirrored
RAID 53Performance + RedundancyMost common because of blend of benefits
RAID 64Performance + RedundancyTwo disk can fail at the same time. Clearly more expensive. Most reliable if you really think two disk can fail at once.
RAID 104Performance + RedundancyCombo of RAID 0 and 1. One disk can fail.

The most common RAID in our opinion is RAID 5, because it uses the least amount of disk but offers redundancy and performance benefits. Please comment if you have an opinion about this.

Checking your system for RAID

Let’s move on to a few commands that allows one to check what RAID controller is installed.

Here we are checking four systems named P, 7, M, and W.

# lspci | grep -i raid
02:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2108 [Liberator] (rev 05)
# lspci | grep -i raid
17:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS-3 3108 [Invader] (rev 02)
# lspci | grep -i raid
03:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS-3 3108 [Invader] (rev 02)
# lspci | grep -i raid
03:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2108 [Liberator] (rev 05)

For reference, all four these servers were ordered across two different data centres over a long period of time.

From this output we can learn a few things:

  • All cards are Broadcom LSI MegaRAID products
  • Two are “Liberator” models 2108 and two are “Invader” models 3108

Communicating with the RAID cards

Next you would want to use a hardware utility to interrogate the drives. This is where is gets very confusing. The utility is called “MegaCLI” and finding the current version can be a nightmare. To make it easy, here is the link:

https://www.broadcom.com/support/download-search?dk=megacli

As of 06 Apr 2023 this search products no less than 62 results. You have to expand the Management Software and Tools section as per the screenshot below:

Yep. I made a square around the date. Imagine, it’s 2023 and you’re working on your mission critical hardware but you have to download a driver dated from 2014. This is really true, have a big swig of coffee.

Next Broadcom have made it really difficult to download, you can just click the link, you have to accept terms. So if you just want to wget it from a server, you can’t. Thanks Broadcom.

Then you have to decompress the files.

Then your path for Linux (RPM based) is easy, but Ubuntu is more complicated.

sudo alien -k --scripts MegaCli-8.07.14-1.noarch.rpm

Potential Errors

On some systems you might encounter some of  these errors:

# /opt/MegaRAID/MegaCli/MegaCli64 
/opt/MegaRAID/MegaCli/MegaCli64: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

It appears that MegaRAID from 2014 uses ncurses 5, not 6:

# rpm -qi ncurses
Name : ncurses
Version : 6.1

CentOS RPM instructions

# rpm -Uvh MegaCli-8.07.14-1.noarch.rpm 
Verifying... ################################# [100%]
Preparing... ################################# [100%]
package MegaCli-8.07.14-1.noarch is already installed

Reference

https://phoenixnap.com/kb/how-to-set-up-hardware-raid-megacli

Share this article

Leave a Reply

Your email address will not be published. Required fields are marked *

Scroll to Top