jump to navigation

CentOS 7 – Installing 2 x 4TB Software RAID LVM GPT BIOSBOOT 20 August 2017

Posted by Maulvi Bakar in : Linux,System,Work , add a comment

Recently, I have an interesting challenge – Installing CentOS 7 on a physical server with 2 x 4TB HDDs!  :-

Anytime you’re playing HDDs larger than 2TBs, you’ll need GPT and  BIOSBOOT is needed. Unless you have deep pockets to pay for hardware RAID, which can simplify your configuration, the poor man’s software RAIID is the way to go.  The catch is that you have to configure almost every aspect of the system with it.

On hardware RAID, once configured in the background, the OS will see only a single HDD Volume for which the OS will be installed onto.  The software RAID depends on the OS. The OS will see all the HDDs and will have to consider them all before Software RAID can be configured.

GPT BIOSBOOT partitions does not work on Software RAID.

It has to be installed in a standard partition – eg. /dev/sda1 – 1MB Size

Consider this scenario – 2 x HDDs, the first HDD will have the following –

/dev/sda1 – BIOSBOOT
/dev/sda2 – MD0 – /boot
/dev/sda3 – MD1 – LVM – swap, /

/dev/sdb1 – MD0 – /boot
/dev/sdb2 – MD1 – LVM – swap, /

Because, the above scenario is what the CentOS 7 GUI Installer allows you to configure.  The obvious solution to this is to have another BIOSBOOT partition on /dev/sdb but the GUI Installer logic (dessign flaw?) won’t allow you to do so.

Obvious weakness will be – what if /dev/sda fails and since only /dev/sdb exists in the RAID Array, without a BIOSBOOT partition, the system will not be bootable.

To overcome this, you’ll need to force the system into having a BIOSBOOT partitions on all HDDs, to ensure the system will bootable, regardless which device go missing. Consider the following –

/dev/sda1 – BIOSBOOT
/dev/sda2 – MD0 – /boot
/dev/sda3 – MD1 – LVM – swap, /

/dev/sdb1 – BIOSBOOT
/dev/sdb2 – MD0 – /boot
/dev/sdb3 – MD1 – LVM – swap, /

To achieve the above ideal layout, you will need to do the following –

  1. Boot from CentOS 7/RHEL 7 Installer
  2. Prep your installation – Date & Time Timezone, Network & Hostname, etc.
  3. Select Installation Detination
  4. Ensure both both disks are selected and Other Storage Options – “I will configure partitioning” radio button is selected
  5. Temporarily go to shell prompt by pressing -> Ctrl + Alt + F2
  6. Execute “gdisk /dev/sda”. I GPT is not available, create it.
  7. Create the BIOSBOOT partition on /dev/sda
  8. n (new) -> Partition = 1 -> Start = Enter -> End = +1M ->Type: EF02 -> w (write)
  9. Repeat the last three steps for /dev/sdb
  10. Once done, return back to GUI Installer by pressing -> Ctrl + Alt + F6
  11. Click the refresh button – Observe that the two Bios Boot partitions will be under ‘Unknown’
  12. You can now proceed with the rest of the partition configuration –
  13. Create the /boot RAID Partition, once created, you will notice one of the BIOSBOOT partition create earlier will appearon the main System partitions list.
  14. Create the swap and “/” LVM partitions, click the “modify” button to RAID it.
  15. Click Done.

Your installation should proceed and BIOSBOOT partitions will be on both disks!

LVM: Duplicate VGName – Howto Rename 31 May 2016

Posted by Maulvi Bakar in : Linux,System,Unix , add a comment

xsshot4d33ca58bce04.png.pagespeed.gp+jp+jw+pj+js+rj+rp+rw+ri+cp+md.ic.ECGX3Ynr_FI have an interesting problem. One of my Virtual Machine was compromised with a trojan/virus. The VM was relatively new with no exposed services to public facing interface except for SSH. It was a CentOS 7 , newly installed with nothing configured on it. I did made sure to update it to the latest updates, thinking it was just fine, I let it at default which by vendor’s default settings – only the SSH is enabled.

A few days later, the VM was compromised.  The tell-tale sign?  The VM was extremely sluggish and my home internet connectivity was sluggish as well. A simple “top” command shows that there are processes running in the background consuming obscene amount of available resources. It seems that this troan/virus is conducting a DDOS uti;izing my resources turning my VM into into a mindless zombie botnet. The process-name? It was randomly generated name. Killing and deleting them will only cause them to regenerate itself.

My first reaction was to take the VM offline.

Since this is a new system, I find it easier to rebuild the system from scratch. I took the VM’s existing disk-image offline and assigned a new one to itand proceeded to install the OS onto it.

Once done, I re-attached the old disk-image back to the VM.

As I rebuild the VM’s OS using the same details, unfortunately, the old disk-image and the new disk-image shares the same LVM Volume Group Name. While I was lucky the VM boots-up finne, I supposed it is due to the new disk-image being the first diskdetected by the OS, mounts fine but the old disk-image’s Volume Group was not.

The “lvcan” commands shows the Volume was inactive –
# lvscan
ACTIVE ‘/dev/centos_vg01/swap’ [2.00 GiB] inherit
ACTIVE ‘/dev/centos_vg01/root’ [7.53 GiB] inherit
inactive ‘/dev/centos_vg01/swap’ [1.00 GiB] inherit
inactive ‘/dev/centos_vg0/root’ [8.47 GiB] inherit

Using the command “vgdisplay”, I acquire the UUID of the Volume Group that I wanted to rename, the it is a simple command of “vgrename”
# vgrename gUgqiK-6KKe-DH70-5Zsr-KMu6-ooQm-BSwlJB centos_vg01_old
Volume group “centos_vg01” successfully renamed to “centos_vg01_old”

Upon the next reboot, both Volume Groups were detected normally –
# lvscan
ACTIVE ‘/dev/centos_vg01/swap’ [2.00 GiB] inherit
ACTIVE ‘/dev/centos_vg01t/root’ [7.53 GiB] inherit
ACTIVE ‘/dev/centos_vg01_old/swap’ [1.00 GiB] inherit
ACTIVE ‘/dev/centos_vg01_old/root’ [8.47 GiB] inherit

Enjoy!

amavis on non-existent ipv6 error 14 January 2015

Posted by Maulvi Bakar in : Linux,System , add a comment

amavis-2A weird issue became noticeable when I no longer receiving mails.

Apparently all mails are stuck in the Postfix ‘mailq’. Tell tale clue was –

Jan 13 07:15:23 www postfix/error[13255]: 08EAA1AD2C: to=<<my email addr-removed!>>, relay=none, delay=63574, delays=63572/1.2/0/0.26, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)

Sure enough, port 10024 was not active. Amavisd is dead!

Enabling debug logging shows below –


Jan 13 07:54:05 www amavis[16106]: Net::Server: Binding to TCP port 10024 on host 127.0.0.1 with IPv4
Jan 13 07:54:05 www amavis[16106]: Net::Server: Binding to TCP port 10024 on host ::1 with IPv6
Jan 13 07:54:05 www amavis[16106]: (!)Net::Server: 2015/01/13-07:54:05 Can't connect to TCP port 10024 on ::1 [Cannot assign requested address]\n at line 68 in file /usr/share/perl5/vendor_perl/Net/Server/Proto/TCP.pm
Jan 13 07:54:05 www amavis[16106]: Net::Server: 2015/01/13-07:54:05 Server closing!

No, I disabled ipv6 for my systems.

How do I ensure ipv6 is not enabled in amavisd ?

echo '$inet_socket_bind = '127.0.0.1';' >> /etc/amavisd/amavisd.conf

A restart of Amavisd is required after that.

DISCLAIMER: Your mileage vary..

Enjoy!

Apache – Directory Listings and Non-Listing Exclusions 5 July 2013

Posted by Maulvi Bakar in : Linux,System,Unix , add a comment

Apache HTTP Server

To display directory listings in a folder, the ‘Options‘ directive is ‘Indexes‘. Usually we include ‘FollowSymLinks‘ as well so that Symbolic Links specified on OS level are honoured. The code syntax is as follows –

Options Indexes FollowSymLinks
IndexIgnore .htacccess privatefolder

Notice that there’s the ‘IndexIgnore‘ directive. This directive will tell apache which files or directories to ignore when listing.

Enjoy!

Lord of the OS 29 May 2013

Posted by Maulvi Bakar in : Linux,Society,System,Unix , add a comment

Recently I visited a friend who is an average computer user. As we were talking he mentioned that he had recently installed Windows 8 on his PC. He told me how happy he was with this operating system and showed me the Windows 8 DVD. To his surprise I threw it into his microwave oven and turned it on. Instantly he got very upset, because the DVD had become precious to him, but I told him: ‘Do not worry, it is unharmed.’ After a few minutes I took the CD out, gave it to him and said: ‘Take a close look at it.’ To his surprise the DVD was quite cool to hold and he remarked that it seemed to be heavier than before. At first he could not see anything, but on the inner edge of the central hole he saw an inscription, an inscription finer than anything he had ever seen before. The inscription shone piercingly bright, and yet remote, as if out of a great depth:

12413AEB2ED4FA5E6F7D78E78BEDE820945092OF923A40EElOE5 I OCC98D444AA08EI

‘I cannot understand these fiery letters,’ he said in a timid voice.
‘No but I can,’ I said. ‘The letters are HEX, of an ancient mode, but the language is that of Microsoft, which I shall not utter here. But in common English this is what it says:

One OS to rule them all, One OS to find them,
One OS to bring them all and in the darkness bind them

It is only two lines from a verse long known in System-lore:

Three OS’s from corporate-kings in their towers of glass,
Seven from valley-lords where orchards used to grow,
Nine from dotcoms doomed to die,
One from the Dark Lord Gates on his dark throne
In the Land of Redmond where the Shadows lie.
One OS to rule them all, One OS to find them,
One OS to bring them all and in the darkness bind them,
In the Land of Redmond where the Shadows lie.

dracut Warning: No root device “block:/dev/disk/by-label/root” found – CentOS 6 19 May 2013

Posted by Maulvi Bakar in : Linux,System,Work , add a comment

I’ve been on GE-mode for the past few months. In effort to maintain stability, I resisted upgrading the KVM-host machine to the latest kernel. Sorry guys, I’m still on CentOS, not on OEL, so no Ksplice!

To cut story short, the kernel upgrades results in the infamous CentOS/RHEL depmod bug! –

dracut Warning: No root device "block:/dev/disk/by-label/root" found
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.32-358.6.2.el6.x86_64 #1
Call Trace:
[] ? panic+0x78/0x143
[] ? do_exit+0x852/0x860
[] ? fput+0x25/0x30
[] ? do_group_exit+0x58/0xd0
[] ? sys_exit_group+0x17/0x20
[] ? system_call_fastpath+0x16/0x1b

The system kernel-panics!

Even the built-in kernel fallback options in GRUB did not activate!!

Canvassing the net suggests that the kernel modules did not install properly and the solution is to install manually via the commands below –

# depmod -ae -F /boot/System.map-2.6.32-358.6.2.el6.x86_64 2.6.32-358.6.2.el6.x86_64
# dracut --force '' 2.6.32-358.6.2.el6.x86_64

I tried that via CentOS Rescue Mode and ‘chroot /mnt/sysimage’. This too was fun, KVM Console ISO Image redirection from your local PC to a site on the far side of the galaxy is something you should only attempt on a high-bandwidth! UniFi is crap!

Luckily the KVM Console supports mounting ISO Image from a SAMBA Share. A small tiny low-end-box serves the purpose nicely and being much closer in peering to the main KVM-host site makes it even better!

Unfortunately, that doesn’t seem to work for me.

But what does work – instead of ‘yum update’, I did a ‘yum localinstall’. Downloaded the kernel and kernel-devel RPMs onto the local disk and installed it after removing the broken kernel before that.

That seems to do the trick!

Hope it solves someone else issue with this.

God willing!

Centos 6.3 – D-Link DWM-652 3G HSDPA Modem – DiGi 3 February 2013

Posted by Maulvi Bakar in : Linux,System , add a comment

Recently I had a need to have a CentOS 6.3 Server connected to the Internet. So, I dug out my old trusty D-Link DWM-652 3G HSDPA Modem and subscribed to a Data-Only DiGi Broadband Internet.

The next challenge is to configure it.

Fortunately, CentOS 6.3 and its most recent kernel – 2.6.32-279.19.1 (at the time of writing) seems modern enough, that it detects the built-in cd-rom storage of the D-Link DWM-652 and promptly ejects it and re-detects the GSM Modem and properly mounts it.

Dmesg Output –
# dmesg
much truncated
usb 5-4: new full speed USB device number 2 using ohci_hcd
usb 5-4: New USB device found, idVendor=1186, idProduct=3e02
usb 5-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 5-4: Product: USB Mass Storage
usb 5-4: Manufacturer: D-Link Corporation
usb 5-4: SerialNumber: 000000000002
usb 5-4: configuration #1 chosen from 1 choice
Initializing USB Mass Storage driver...
scsi6 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 2
usb-storage: waiting for device to settle before scanning
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usb-storage: device scan complete
scsi 6:0:0:0: CD-ROM D-Link CDROM 1.01 PQ: 0 ANSI: 4
scsi 6:0:0:0: Attached scsi generic sg2 type 5
sr0: scsi3-mmc drive: 0x/0x caddy
Uniform CD-ROM driver Revision: 3.20
sr 6:0:0:0: Attached scsi CD-ROM sr0
sr0: CDROM (ioctl) error, command: Get configuration 46 00 00 00 00 00 00 00 20 00
sr: Sense Key : Hardware Error [current]
sr: Add. Sense: No additional sense information
sr0: CDROM (ioctl) error, command: Get configuration 46 00 00 00 00 00 00 00 20 00
sr: Sense Key : Hardware Error [current]
sr: Add. Sense: No additional sense information
sr0: CDROM (ioctl) error, command: Get configuration 46 00 00 00 00 00 00 00 20 00
sr: Sense Key : Hardware Error [current]
sr: Add. Sense: No additional sense information
usb 5-4: USB disconnect, device number 2
Buffer I/O error on device sr0, logical block 1
Buffer I/O error on device sr0, logical block 512
Buffer I/O error on device sr0, logical block 7
Buffer I/O error on device sr0, logical block 1
Buffer I/O error on device sr0, logical block 512
usb 5-4: new full speed USB device number 3 using ohci_hcd
usb 5-4: New USB device found, idVendor=1186, idProduct=3e04
usb 5-4: New USB device strings: Mfr=1, Product=2, SerialNumber=20
usb 5-4: Product: D-Link DWM-652 3.5G HSDPA Adapter
usb 5-4: Manufacturer: D-Link Corporation
usb 5-4: SerialNumber: 0000000000000000
usb 5-4: configuration #1 chosen from 1 choice
usb-storage: device ignored
usb-storage: device ignored
usb-storage: device ignored
usbcore: registered new interface driver usbserial
USB Serial support registered for generic
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial Driver core
USB Serial support registered for GSM modem (1-port)
option 5-4:1.1: GSM modem (1-port) converter detected
usb 5-4: GSM modem (1-port) converter now attached to ttyUSB0
option 5-4:1.2: GSM modem (1-port) converter detected
usb 5-4: GSM modem (1-port) converter now attached to ttyUSB1
usbcore: registered new interface driver option
option: v0.7.2:USB Driver for GSM modems

Note the USB Device’s ttyUSB which you may need to guess the proper ones. Usually if you do not have any other USB Devices connected to it, it’ll be the ttyUSB0 which you will need as the modem entry for the /etc/wvdial.conf file.

# ls -l /dev/ttyUSB*
crw-rw----. 1 root dialout 188, 0 Feb 3 02:16 /dev/ttyUSB0
crw-rw----. 1 root dialout 188, 1 Feb 3 00:36 /dev/ttyUSB1

Next is to install the necessary RPM Packages, in this case – wvdial

# yum install wvdial

You’ll notice that it installs the dependencies –

================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
wvdial x86_64 1.60-12.el6 base 92 k
Installing for dependencies:
libwvstreams x86_64 4.6-6.el6 base 603 k
lockdev x86_64 1.0.1-18.el6 base 25 k
ppp x86_64 2.4.5-5.el6 base 323 k

Below is the contents that I used in the file /etc/wvdial.conf. Take note of the Modem entry and its tty device.

# cat /etc/wvdial.conf
[Dialer Defaults]
Modem = /dev/ttyUSB0
Modem Type = Analog Modem
ISDN = 0
Baud = 460800
Dial Attempts = 1
Username = user
Password = pass
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 = AT+CGDCONT=1,"IP","DiGi.internet"
Phone = *99#
Stupid Mode = 1

Connecting –

# wvdial Defaults
--> WvDial: Internet dialer version 1.60
--> Cannot get information for serial port.
--> Initializing modem.
--> Sending: ATZ
ATZ
OK
--> Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
OK
--> Sending: AT+CGDCONT=1,"IP","DiGi.internet"
AT+CGDCONT=1,"IP","DiGi.internet"
OK
--> Modem initialized.
--> Sending: ATDT*99#
--> Waiting for carrier.
ATDT*99#
CONNECT
--> Carrier detected. Starting PPP immediately.
--> Starting pppd at Sun Feb 3 01:11:03 2013
--> Pid of pppd: 29518
--> Using interface ppp0
--> local IP address xxx.xxx.xxx.xxx
--> remote IP address 10.64.64.64
--> primary DNS address xxx.xxx.xxx.xxx
--> secondary DNS address xxx.xxx.xxx.xxx

Note – remote IP is the ISP’s Gateway IP, you may need to alter your current Default Gateway to this one –

Make note of the current Default Gateway, if it is an IP Address –

# netstat -r|grep default | awk {'print $2'}

If it is a hostname –

# netstat -r|grep default | awk {'print $2'}| awk -F. '{ print $1 }'

Delete the existing –

# route del default

Add new one –

# route add -net default gw 10.64.64.64 dev ppp0

To disconnect, simply press CTRL-C

^CCaught signal 2: Attempting to exit gracefully...
--> Terminating on signal 15
--> Connect time 21.4 minutes.
--> Disconnecting at Sun Feb 3 01:32:28 2013

Once disconnected, you may want to revert back to the previous Default Gateway. Use below command to add it back –

# route add -net default your-gateway

Centos Public Mirror – http://centos.maulvi.net 15 October 2009

Posted by Maulvi Bakar in : Linux,Society,System , 1 comment so far

To the world..

I present the Centos Public Mirror hosted on http://centos.maulvi.net

Basically, I’ve been installing and re-installing centos over and over and over..  Also, at any one time, I have a bunch of centos boxes running both physically and virtually.  This prompts me to run my own centos mirror.  It’s twofold – it saves time and make it easier for me to simply grab any packages.

Then I had a revelation – after leeching from the community, it’s time to give back something… This mirror is one small way I could do to give back and with a big THANK YOU to the wonderful centos linux community.

Enjoy!

Creating a new Linux RAID1 device and extending an existing LVM Volume onto it. RHEL/Centos 5 13 June 2009

Posted by Maulvi Bakar in : Linux,System , add a comment

I’m seriously into server consolidation and virtualization.  I’m going to eliminate my other VMWare server and just concentrate on one.

First order of the day is to upgrade the RAM and Available HDD space. Upgrading the RAM is simple enough, just add/replace the modules.

Expanding the available space is another matter.  This is actually an extension of my previous posting – New HDD, enlarging Red Hat/Centos ext3/lvm partition, but there’s another factor in the equation, RAID1 (Mirroring) devices.

The existing root partition resides in an LVM partition which in turn resides in a RAID1 (Mirroring) Multi-Disk (MD) partition which is based on a twin 400GB HDD configuration.

I’ve acquired a pair of 500GB HDDs. Now to extend the existing LVM ontothe newly acquired disks in a RAID1 configuration.

Step 1 is to create Software RAID partitions on the said disks –
# fdisk /dev/sdc
# fdisk /dev/sdd

Create a new ‘sdc1’ and ‘sdd1’ partitions using type FD, which is Linux RAID Autodetect.

Next, we need to create the Multi-Disk Volume –
# mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
Since I already have ‘md0’ for my ‘/boot’ and ‘md1’ is the existing volume with the existing LVM that was the intended target for expansion, thus naturally I’m using ‘md2’.

Basically, I am creating ‘md2’ device in RAID1 configuration using 2 devices namely ‘/dev/sdc1’ and ‘/dev/sdd1’.

You can monitor the status of your RAID devices using ‘/proc/mdstat’ –
# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdd1[1] sdc1[0]
487331648 blocks [2/2] [UU]
[>....................] resync = 1.9% (9622656/487331648) finish=90.0min speed=88417K/sec

The steps here onwards are basically the same as previously mentioned here, the only difference is the ‘/dev/md2’ device is the target.

To start, we need to create a Physical Volume within the newly created ‘/dev/md2’ RAID device.
# pvcreate /dev/md2

After that we will extend the existing volume ‘VolGroup00? onto the newly created physical volume.
# vgextend VolGroup00 /dev/md2

Once done, the next step is to extend the Logical Volume within the volume group to use the free space newly made available when you extend the volume group previously.
# lvextend /dev/VolGroup00/LogVol00 /dev/md2
Here is the difference where I did not specify the space size to extend as the default will be to use all available spaces.

And finally, we’ll enlarge the ext3 partition to make use of the newly available free space in the logical volume.
# resize2fs /dev/VolGroup00/LogVol00

Enjoy!

New HDD, enlarging Red Hat/Centos ext3/lvm partition 16 December 2007

Posted by Maulvi Bakar in : Home,Linux,System , 3 comments

Hmmm,

Suddenly, I’ve run out of space on one of my servers at home. Solution, add a new harddisk, extend existing partition onto the new harddisk.. Simple right? Right…

Firstly, fix the new harddisk onto the machine. Fdisk it like thus –

# fdisk /dev/sdb

Create a new ‘sdb1’ partition using type 8e, which is Linux LVM.

Next, we need to create a Physical Volume within the newly created sdb1 partition.

# pvcreate /dev/sdb1

After that we will extend the existing volume ‘VolGroup00’ onto the newly created physical volume.

# vgextend VolGroup00 /dev/sdb1

Once done, the next step is to extend the Logical Volume within the volume group to use the free space newly made available when you extend the volume group previously.

# lvextend -L 40G /dev/VolGroup00/LogVol00

And finally, we’ll enlarge the ext3 partition to make use of the newly available free space in the logical volume.

# resize2fs /dev/VolGroup00/LogVol00

Enjoy!

Typo error since fixed as advised! Thank you!

π