HOW TO RECOVER DELETED ORACLE DATAFILES WITH NO DOWNTIME

April 11th, 2021, posted in Oracle Queries
Share

So you have accidentally removed a datafile from your production database? First thing, DON’T PANIC !! There’s an easy way to recover deleted datafiles, for as long as your database remains up. The procedure below works on linux, however this method conceivably can work for other platforms.

This procedure will even work if your database is in NOARCHIVELOG mode.

You may have reached this posting through Google, and in a rush to get the solution right away, so here it is.

The recovery is done in two phases.

Phase 1: instant recovery to prevent Oracle errors (like ORA-27041 “unable to open file”, ORA-01110, ORA-01116)

  1. Find the PID of DBWRITER for the right database.
    ps -ef | grep dbw0_SID_OF_YOUR_DB
    oracle   12272     1  0 10:55 ?        00:00:00 ora_dbw0_test
    oracle   12941 11501  0 12:36 pts/0    00:00:00 grep dbw0_test
  2. List the deleted file handles for that DBWRITER process.
    ls -l /proc/_PID_OF_DBWRITER/fd | grep deleted
    
    lrwx------  1 oracle oinstall 64 Oct 15 11:24 10 -> /home/oracle/product/10.2.0.2/dbs/lkinsttest (deleted)
    lrwx------  1 oracle oinstall 64 Oct 15 11:24 23 -> /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k6xzjpm_.dbf (deleted)
  3. Create a symbolic link to your datafile with the original name.
    ln -s /proc/PID_OF_DBWRITER/fd/23 /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k6xzjpm_.dbf

    That’s all. Now you are no longer going to get errors. However, if your database goes down now, you will lose that datafile for good.

Phase 2: restore the file

ARCHIVELOG database

  1. (Optional.) Issue a checkpoint. This is to reduce the recovery time when bringing the file online, depending on activity for that datafile. Unfortunately, you can’t checkpoint a single file, so the checkpoint may take some time.
    alter system checkpoint;
  2. Backup the datafile with rman. Why rman? It’s much easier then you think. Total downtime is about one second for inactive datafiles, and more for active ones (with writes).
    rman target /
    report schema;
    backup as copy datafile YOUR_DATAFILE_NUMBER format '/location_of_your_database/new_name_for_File.dbf';
    sql 'alter database datafile YOUR_DATAFILE_NUMBER offline';
    switch datafile YOUR_DATAFILE_NUMBER to copy;
    recover datafile YOUR_DATAFILE_NUMBER;
    sql 'alter database datafile YOUR_DATAFILE_NUMBER online';
    exit;

NOARCHIVELOG database

  1. Make the tablespace with that datafile read only
    select distinct tablespace_name from dba_data_files where file_name = 'YOUR_DELETED_FILE';
    alter tablespace THE_TABLESPACE read only;
  2. Copy the file from the symlink to a new name
    cp SIM_LINK_DATA_FILE NEW_NAME_FOR_DATAFILE.dbf
  3. WARNING: Ensure your copy is complete! Then, crash the database.
    /*WAIT FOR COPY!!!*/
    shutdown abort;
  4. Remove the now invalid symlink, and rename the datafile to its original name. Be careful not to remove the wrong file now — that would be a disaster:
    rm -i SIM_LINK_DATA_FILE
    mv NEW_NAME_FOR_DATAFILE.dbf SIM_LINK_DATA_FILE
  5. Startup your database normally and make the tablespace read/write.
    startup
    alter tablespace THE_TABLESPACE read write;

I hope this helps you to get out of a nasty situation.

Here’s both cases fully captured from terminal. Note, I am using Oracle-managed files. This doesn’t change the steps.

/ra5a/orabkp/test/TEST/datafile> sqlplus '/ as sysdba'

SQL*Plus: Release 10.2.0.2.0 - Production on Mon Oct 15 12:31:55 2007

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

SQL> 
SQL> select tablespace_name, file_name from dba_data_files

TABLESPACE_NAME                FILE_NAME
------------------------------ ------------------------------------------------------------
USERS                          /ra5a/orabkp/test/TEST/datafile/o1_mf_users_3k6xgwhb_.dbf
SYSAUX                         /ra5a/orabkp/test/TEST/datafile/o1_mf_sysaux_3k6xgwdf_.dbf
UNDOTBS1                       /ra5a/orabkp/test/TEST/datafile/o1_mf_undotbs1_3k6xgwg9_.dbf
SYSTEM                         /ra5a/orabkp/test/TEST/datafile/o1_mf_system_3k6xgwd4_.dbf
EXAMPLE                        /ra5a/orabkp/test/TEST/datafile/o1_mf_example_3k6xjdjw_.dbf
LOST                           /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf

6 rows selected.

SQL> select tablespace_name, table_name from dba_tables where owner = 'TESTING';

TABLESPACE_NAME                TABLE_NAME
------------------------------ ------------------------------
LOST                           LOST_TABLE

SQL> connect testing/testing
SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> alter system flush buffer_cache;

System altered.

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> Disconnected from Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

/ra5a/orabkp/test/TEST/datafile> ls -lF
total 1015132
-rw-r-----  1 oracle oinstall 157294592 Oct 15 12:22 o1_mf_example_3k6xjdjw_.dbf
-rw-r-----  1 oracle oinstall 104865792 Oct 15 12:22 o1_mf_lost_3k74mq08_.dbf
-rw-r-----  1 oracle oinstall 241180672 Oct 15 12:32 o1_mf_sysaux_3k6xgwdf_.dbf
-rw-r-----  1 oracle oinstall 503324672 Oct 15 12:32 o1_mf_system_3k6xgwd4_.dbf
-rw-r-----  1 oracle oinstall  20979712 Oct 15 10:17 o1_mf_temp_3k6xj9xn_.tmp
-rw-r-----  1 oracle oinstall  26222592 Oct 15 12:32 o1_mf_undotbs1_3k6xgwg9_.dbf
-rw-r-----  1 oracle oinstall   5251072 Oct 15 12:22 o1_mf_users_3k6xgwhb_.dbf

/ra5a/orabkp/test/TEST/datafile> rm o1_mf_lost_3k74mq08_.dbf
/ra5a/orabkp/test/TEST/datafile> sqlplus testing/testing

SQL*Plus: Release 10.2.0.2.0 - Production on Mon Oct 15 12:35:24 2007

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> alter system flush buffer_cache;

System altered.

SQL> select count(*) from lost_table;
select count(*) from lost_table
*
ERROR at line 1:
ORA-01116: error in opening database file 6
ORA-01110: data file 6:
'/ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf'
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

SQL> 
SQL> Disconnected from Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

/ra5a/orabkp/test/TEST/datafile> ps -ef|grep dbw0_test
oracle   12272     1  0 10:55 ?        00:00:00 ora_dbw0_test
oracle   12941 11501  0 12:36 pts/0    00:00:00 grep dbw0_test
/ra5a/orabkp/test/TEST/datafile> ls -l /proc/12272/fd|grep deleted
lrwx------  1 oracle oinstall 64 Oct 15 11:24 10 -> /home/oracle/product/10.2.0.2/dbs/lkinsttest (deleted)
lrwx------  1 oracle oinstall 64 Oct 15 12:17 26 -> /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf (deleted)

/ra5a/orabkp/test/TEST/datafile> ln -s /proc/12272/fd/26 /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf

/ra5a/orabkp/test/TEST/datafile> ls -lF
total 912620
-rw-r-----  1 oracle oinstall 157294592 Oct 15 12:22 o1_mf_example_3k6xjdjw_.dbf
lrwxrwxrwx  1 oracle oinstall        17 Oct 15 12:37 o1_mf_lost_3k74mq08_.dbf -> /proc/12272/fd/26
-rw-r-----  1 oracle oinstall 241180672 Oct 15 12:32 o1_mf_sysaux_3k6xgwdf_.dbf
-rw-r-----  1 oracle oinstall 503324672 Oct 15 12:32 o1_mf_system_3k6xgwd4_.dbf
-rw-r-----  1 oracle oinstall  20979712 Oct 15 10:17 o1_mf_temp_3k6xj9xn_.tmp
-rw-r-----  1 oracle oinstall  26222592 Oct 15 12:32 o1_mf_undotbs1_3k6xgwg9_.dbf
-rw-r-----  1 oracle oinstall   5251072 Oct 15 12:22 o1_mf_users_3k6xgwhb_.dbf

/ra5a/orabkp/test/TEST/datafile> sqlplus testing/testing

SQL*Plus: Release 10.2.0.2.0 - Production on Mon Oct 15 12:38:18 2007

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> alter system flush buffer_cache;

System altered.

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> 
SQL> Disconnected from Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

/ra5a/orabkp/test/TEST/datafile> rman target /

Recovery Manager: Release 10.2.0.2.0 - Production on Mon Oct 15 12:39:48 2007

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

connected to target database: TEST (DBID=1934173752)

RMAN> report schema;

using target database control file instead of recovery catalog
Report of database schema

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    480      SYSTEM               ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_system_3k6xgwd4_.dbf
2    25       UNDOTBS1             ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_undotbs1_3k6xgwg9_.dbf
3    230      SYSAUX               ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_sysaux_3k6xgwdf_.dbf
4    5        USERS                ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_users_3k6xgwhb_.dbf
5    150      EXAMPLE              ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_example_3k6xjdjw_.dbf
6    100      LOST                 ***     /ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    20       TEMP                 32767       /ra5a/orabkp/test/TEST/datafile/o1_mf_temp_3k6xj9xn_.tmp

RMAN> backup as copy datafile 6 format '/ra5a/orabkp/test/TEST/datafile/lost.dbf';

Starting backup at 2007-10-15 12:40:45
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=154 devtype=DISK
channel ORA_DISK_1: starting datafile copy
input datafile fno=00006 name=/ra5a/orabkp/test/TEST/datafile/o1_mf_lost_3k74mq08_.dbf
output filename=/ra5a/orabkp/test/TEST/datafile/lost.dbf tag=TAG20071015T124045 recid=13 stamp=636036046
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:01
Finished backup at 2007-10-15 12:40:46

RMAN> sql 'alter database datafile 6 offline';

sql statement: alter database datafile 6 offline

RMAN> switch datafile 6 to copy;

datafile 6 switched to datafile copy "/ra5a/orabkp/test/TEST/datafile/lost.dbf"

RMAN> recover datafile 6;

Starting recover at 2007-10-15 12:41:07
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 2007-10-15 12:41:07

RMAN> sql 'alter database datafile 6 online';

sql statement: alter database datafile 6 online

RMAN> 

Recovery Manager complete.

/ra5a/orabkp/test/TEST/datafile> ls -lF
total 1015132
-rw-r-----  1 oracle oinstall 104865792 Oct 15 12:41 lost.dbf
-rw-r-----  1 oracle oinstall 157294592 Oct 15 12:22 o1_mf_example_3k6xjdjw_.dbf
lrwxrwxrwx  1 oracle oinstall        17 Oct 15 12:37 o1_mf_lost_3k74mq08_.dbf -> /proc/12272/fd/26
-rw-r-----  1 oracle oinstall 241180672 Oct 15 12:32 o1_mf_sysaux_3k6xgwdf_.dbf
-rw-r-----  1 oracle oinstall 503324672 Oct 15 12:38 o1_mf_system_3k6xgwd4_.dbf
-rw-r-----  1 oracle oinstall  20979712 Oct 15 10:17 o1_mf_temp_3k6xj9xn_.tmp
-rw-r-----  1 oracle oinstall  26222592 Oct 15 12:38 o1_mf_undotbs1_3k6xgwg9_.dbf
-rw-r-----  1 oracle oinstall   5251072 Oct 15 12:22 o1_mf_users_3k6xgwhb_.dbf

/ra5a/orabkp/test/TEST/datafile> rm o1_mf_lost_3k74mq08_.dbf
/ra5a/orabkp/test/TEST/datafile> sqlplus testing/testing

SQL*Plus: Release 10.2.0.2.0 - Production on Mon Oct 15 12:42:03 2007

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

Connected to:
Oracle Database 10g Release 10.2.0.2.0 - 64bit Production

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070

SQL> alter system flush buffer_cache;

System altered.

SQL> select count(*) from lost_table;

  COUNT(*)
----------
     50070


Share

For My Eyes And Heart Sleep Is Haram

April 4th, 2021, posted in Rumi
Share

Moulana Muhammad Jalaluddin Rumi, rumi,Muhammad Jalaluddin Rumi,persian poetry,poetry,english translation

Moulana Muhammad Jalaluddin Rumi, rumi,Muhammad Jalaluddin Rumi,persian poetry,poetry,english translation

Share

Checking Last 10 Reboots of a Solaris Server

March 28th, 2021, posted in Solaris
Share

 

On your Solaris boxes here, you’d need to doChecking Last 10 Reboots of a Solaris Server,Checking Last 10 Reboots, Solaris Server,Last 10 Reboots of a Solaris Server,Reboots of a Solaris Server,Reboots of a Solaris,Sys DBA,DBA,Reboot
last -10 reboot to actually get the times that the system was rebooted.
As it shows the system down messages, as well as the boot messages.

 

 

[email protected] # last -10 reboot
reboot system boot Sun Aug 16 10:53
reboot system down Sun Jun 14 08:50
reboot system boot Sun Jun 14 08:17
reboot system down Mon Mar 2 12:44
reboot system boot Mon Jan 27 10:23
reboot system down Sun Jan 26 13:15
reboot system boot Sun Jan 26 12:36
reboot system down Wed Jan 22 14:40
reboot system boot Mon Jan 20 13:55
reboot system down Mon Jan 20 12:59
Share

SWITCH BETWEEN ALOM AND ILOM ON SPARC TSERIES SERVER

March 14th, 2021, posted in Solaris
Share

Switching between ALOM and ILOM is applicable on SPARC Servers (T-Series)

SWITCH FROM ILOM TO ALOM :
If you have logged to ilom as a root user (you may be logged in as different user) and you want to switch to ALOM then run this below command, logout and log back in.

->set /SP/users/root cli_mode=alom

SWITCH FROM ALOM TO ILOM :
If you have logged to alom shell as a root user (you may be logged in as different user) and you want to switch to ILOM shell then run the below command, logout and log back in.

sc>userclimode root default
Share

I Will Not Say That Man Of Exalted Station

March 11th, 2021, posted in Rumi
Share

Moulana Muhammad Jalaluddin Rumi, rumi,Muhammad Jalaluddin Rumi,persian poetry,poetry,english translation

 

These couplets praise Rumi (the man of exalted [spiritual] station) and his Masnavi-e Manavi (‘Spiritual Couplets’).

Some Muslims consider the Masnavi second to the Qur’an and Bukhari, and others consider Rumi a heretic and his Masnavi a misguidance.

Moulana Muhammad Jalaluddin Rumi, rumi,Muhammad Jalaluddin Rumi,persian poetry,poetry,english translation

Share