Archive for the ‘TEChNoLoGY’ Category

How To Add-Remove Vdisk From Guest LDOMS

September 8th, 2019, posted in Solaris

I’ll share the simple steps needed to add and remove a virtual disk in a running domain without any outage.
This is a system running Oracle VM Server for SPARC 3.1 with a Solaris 11.1 guest domain named ldom0.
I used NFS storage because it is easy to set up and lets me use live migration.


Adding a virtual disk to a running domain

The entire sequence of commands in the control domain defines, adds and removes a disk while the guest domain runs:

# mkfile -n 20g  /ldomsnfs/ldom0/disk1.img       # 1. create a disk image file
# ldm add-vdsdev /ldomsnfs/ldom0/disk1.img vol01@primary-vds0 # 2. define vdisk
# ldm add-vdisk vdisk01 vol01@primary-vds0 ldom0 # 3. add disk to the domain
# ldm rm-vdisk vdisk01 ldom0                     # 4. take it away from the domain.
# ldm rm-vdsdev vol01@primary-vds0               # 5. undefine the virtual disk
# rm disk1.img                                   # 6. save a little space.

That’s all there is to it. The new disk is available for the domain’s use after step 3 until I take it away in step 4.


Viewing reconfiguration from within the guest

Let’s take a look from the guest domain’s perspective.
In the guest, you can see one disk before adding more (before command 3, above) via the formatcommand:

# format
Searching for disks...done
       0. c3d1 
Specify disk (enter its number): ^C

There’s one disk until ldm add-vdisk is issued in the control domain.
That results in a dynamic reconfiguration event that can be seen, if you are curious, by entering dmesg within the guest:

# dmesg|tail
Nov 20 12:03:23 ldom0 vdc: [ID 625787] vdisk@0 is online using ldc@16,0
Nov 20 12:03:23 ldom0 cnex: [ID 799930] channel-device: vdc0
Nov 20 12:03:23 ldom0 genunix: [ID 936769] vdc0 is /virtual-devices@100/channel-devices@200/disk@0
Nov 20 12:03:23 ldom0 genunix: [ID 408114] /virtual-devices@100/channel-devices@200/disk@0 (vdc0) online

You can see the added disk using format and then use it. In this case I created a temporary ZFS pool.

# format
Searching for disks...done
       0. c3d0 
       1. c3d1 
Specify disk (enter its number): ^C
# zpool create temp c3d0
# zpool list
rpool  19.9G  5.27G  14.6G  26%  1.00x  ONLINE  -
temp   19.9G   112K  19.9G   0%  1.00x  ONLINE  -

At this point I can just go ahead and use the added disk space. I could have done other things like add it to the existing ZFS pool
to make it a mirror, but this illustrates the point.


What happens if I try to remove an in-use disk

It could be very damaging to remove a virtual device while it is in use, so the default behavior
is that Solaris tells logical domains manager that the device is in use and cannot be removed.
That’s a very important advantage of Oracle VM Server for SPARC: the logical domains framework and Solaris work cooperatively,
in this and many other aspects.

In this case, we’re prevented from yanking a disk while it is in use.
If I try to remove the disk while it’s in use, I get an error message – exactly what you want:

# ldm rm-vdisk vdisk01 ldom0
Dynamic reconfiguration of the virtual device on domain ldom0
failed with error code (-122).
The OS on domain ldom0 did not report a reason for the failure.
Check the logs on that OS instance for any further information.
Failed to remove vdisk instance

The reason is “because it’s in use!” 🙂 An administrator would log into the guest to see what file systems are mounted.
This behavior can be overridden using the “-f” option if you are certain
you know what you’re doing.

Removing the disk

I issued zpool destroy temp in the guest and repeated the ldm rm-vdsdev and it worked.
Using zpool export temp would work just as well, and if I choose I can add that virtual disk to a different
domain and it could use zpool import temp to access data created by ldom0.
With other file systems, a regular umount would have the same effect, making it possible to remove the disk without -f.

The format command now shows only one disk again, and dmesg shows kernel messages when disk went offline:

Nov 20 12:42:10 ldom0 vdc: [ID 990228] vdisk@0 is offline
Nov 20 12:42:10 ldom0 genunix: [ID 408114] /virtual-devices@100/channel-devices@200/disk@0 (vdc0) offline



Solaris and the logical domain manager are engineered to work together in a coordinated fashion
to provide operational flexibility.
One of the values this provides is that
administrators can safely add and remove virtual devices while domains run.
This can be used for operational tasks like adding or removing disk capacity or IOPS as needed.
The same capabilities are also available for virtual network devices.


Enable Pluggable Database(PDB) Archivelog Mode

August 26th, 2019, posted in Oracle Queries

Enabling archive log mode -12c pluggable database/Container database

Since the Redologs are created at container database level in 12c and not at pluggable database level.
(Enabling archivelog at pluggable database level is not possible). Archiving is done at CDB’s.

You can check archive log mode either by querying v$database or archive-log list :


SQL> select name,open_mode,log_mode from v$database;

NAME      OPEN_MODE            LOG_MODE
--------- -------------------- ------------


SQL> archive log list
Database log mode        Archive Mode
Automatic archival        Disabled
Archive destination        USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     11
Next log sequence to archive   13
Current log sequence        13

***************  ***************
To enable the Archive-log mode
***************  ***************immam_dba,dba immam,imam dba,dba imam,oracle clone issue,oracle database,oracle application,oracle clone issue,ora oracle

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> startup mount;
ORACLE instance started.

Total System Global Area  788529152 bytes
Fixed Size      2929352 bytes
Variable Size    541068600 bytes
Database Buffers   239075328 bytes
Redo Buffers      5455872 bytes
Database mounted.

Database altered.

Database altered.

SQL> select name,open_mode,log_mode from v$database;

NAME      OPEN_MODE            LOG_MODE
--------- -------------------- ------------

SQL> archive log list
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     413
Next log sequence to archive   415
Current log sequence           415


NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      /u03/app/oracle/fast_recovery_area
db_recovery_file_dest_size           big integer 27G



Apache HTTP_Server – Failed to start a managed process after the maximum retry limit Log (HTTP_Server~1)

August 12th, 2019, posted in Oracle

Oracle HTTP Server, residing on a Unix platform, fails to start using OPMN. For example, OPMN reports the generic error


opmnctl startall
opmnctl: starting opmn and all managed processes...
opmn id=oradb:6200
0 of 1 processes started.


--> Process (pid=24042)
Failed to start a managed process after the maximum retry limit Log::

The HTTP_Server~1 OPMN log file just reports that the HTTP Server has been started, but there are actually no httpd processes present on the system.

There is also nothing written to the $ORACLE_HOME/apache/apache/logs directory even when HTTP Server logLevel is set to debug.

Furthermore there are no core / segmentation fault files created.

In order for the HTTP Server to listen on ports < 1024 e.g 80 and 443 the ownership and permissions of the Oracle HTTP Server binary – ‘.apachectl’ – have been changed as follows:

One possible work-around is to change .apachectl to belong to another group – such as the generic one users (rather than have the file belong to the oracle group – typically called ‘oinstall’) e.g



chown root:users .apachectl
chmod 6750 .apachectl

Show Faulted Hardware in ILOM

August 5th, 2019, posted in Solaris

in an ILOM (Integrated Lights Out Manager). On this page I will use the example of a chassis fan module error. If you follow my notes and the error clears Then you didn’t have a real issue. On the other hand, If after following my notes you can’t clear the error. Then you have a real hardware issue. You can’t clear errors if the error is still an issue.

This is how you login to the command line interface for the ILOM.


man@earth> ssh root@ilom

The command below is one way to show system faults. The only target you should see is shell. If you see anything other then shell it is a fault. In the example below, the ILOM shows a bad system fan. Shown as 0 (/SYS/FMO).


–> show /SP/faultmgmt

0 (/SYS/FM0)



Using the show faulty command is anther way to see the system faults. This command shows a lot more detail. If you have a support contract with Oracle, you will want to paste the output of this command into the ticket, you submit to MOS. The show faulty command can be used without any paths, which will be extra useful if are coming in from a chassis ILOM.


–> show faulty
Target                    | Property                   | Value
/SP/faultmgmt/0    | fru                            | /SYS/FM0
/SP/faultmgmt/0/   | class                         |
faults/0                  |                                  |
/SP/faultmgmt/0/   | sunw-msg-id            | SPX86-8X00-33
faults/0                  |                                  |
/SP/faultmgmt/0/   | component               | /SYS/FM0
faults/0                   |                                 |
/SP/faultmgmt/0/   | uuid                          | 8692c3e4-G481-635e-f8e2-f3f215d1
faults/0                   |                                 | 13f0
/SP/faultmgmt/0/   | timestamp                | 2013-10-02/12:10:43
faults/0                   |                                 |
/SP/faultmgmt/0/   | detector                   | /SYS/FM0/ERR
faults/0                   |                                  |
/SP/faultmgmt/0/   | product_serial_number | 1203FMM107
faults/0                   |                                  |

The command below shows the event log, which will also contain the system hardware errors.


–> show /SP/logs/event/list

To clear the hardware fault from the logs run the command below.


–> show /SP/logs/event/ clear=true

Run this command to clear the fan error.


–> set /SYS/FM0 clear_fault_action=true

Try to clear the hardware fault. If the hardware is really having an issue, the hardware fault will come back. In about a minute or less. If you can’t clear the error and you have a support contract then this is when you summit your ticket.

If you have any questions or I missed something let me know.