Category Archives: Most Popular

ORA-14128: FOREIGN KEY constraint mismatch in ALTER TABLE EXCHANGE PARTITION : The solution

I am writing this blog to provide a simple way  to resolve ORA-14128.    I did not find any reasonable documented  solution to fix this issue w/o disabling the constraints.

Here goes my story.  Upgrading from Oracle 10.2.0.3 to 11.2.0.2 introduced few problems mainly due to our high concurrency and high availability OLTP system.  In 10.2.0.3 , our partition maintenance was done online w/o disabling constraints using the approach described below with an example..

Lets assume tables  transaction and sales  with following attributes:

Transaction (Parent/Child) table with transid as primary key and  index on salesid

.
Sales (Parent) table with salesid as primary key

.
Foreign Key Definition: transaction.salesid  references sales.salesid. Also lot of other tables references transid.transaction. Therefore transaction table is both parent and child.

.

Partition Maintenance Approach: Our ultimate goal for partition maintenance is to drop old partitions.  So all I have  do  to issue “alter table transaction drop partition transaction_part_2011”.  This command will only succeed if there are no foreign key constraints referencing this parent table. Otherwise it will result in “ORA-02266: unique/primary keys in table referenced by enabled foreign keys”.  Since partition maintenance is done online, we don’t have the luxury to disable constraints.

Therefore we had to do all the additional steps involved with partition exchange to accommodate online partition maintenance.  And one of the key requirements of partition exchange is that the tables being exchanged must be identical with respect to structure like same indexes , constraints etc.  You also have the option of disabling constraints to get to this goal.

Note: With steps 1-4 we are trying to make transaction_duplicate table identical to transaction

  1.  create table transaction_Duplicate as select * from Transaction where 1 = 0;
  2. alter table transaction_Duplicate add constraint Transaction_dup_pk primary key(transid);
  3. create index salesid_idx on Transaction_Duplicate(salesid) ;
  4. alter table transaction_Duplicate add constraint fk_salesid foreign key (salesid) references sales(salesid);
  5. alter table transaction exchange partition Transaction_part_2011 with table Transaction_Duplicate including indexes without validation;
  6. alter table transaction drop partition transaction_part_2011;

VERY IMPORTANT: Step-4 is optional if the one of the tables used in exchange is empty which is true in our case as we are creating a dummy duplicate table.

Our problems started with Step-4 after upgrading to 11.2.0.2. We started getting  “ORA-00054:resource busy and acquire with NOWAIT specified” because Oracle changed the locking behavior in 11g.

We opened SR with Oracle and as per Oracle, the code fix for bug 5909305 introduces an intentional change in locking behavior and that change is effective from 11.1.0.6 onwards. ie: From 11g onwards it is correct and expected that DML on the child table will take an SX lock on the parent.

For customers that cannot live with the changed behavior the fix in bug 6117274 allows the change in locking to be reverted to pre-11g behavior by setting “_fix_control” to ‘5909305:OFF’.

.

In our case since Step-4 was optional , we removed step-4 but our problems did not stop there.  Due to nature of our business , every change that goes into the database requires a rollback just in case its required. In other words,  we wanted to rollback the partition  exchange by performing another exchange. Now Step-4 became a must for rollback as both the tables are not empty after the 1st exchange(One of the partition is empty not  the table.).

We were back to square one with small difference. Initially we were trying to fix ORA-00054 and now we are trying to fix ORA-14128. After lot of reading , trials and prototypes, I was able to fix ORA-14128. The solution is very simple. All you had to do was put  the referential integrity constraints in “ENABLE NOVALIDATE“. This was OK for our  database. So if you can put your constraints in  “ENABLE NOVALIDATE” , then you have a simple fix for ORA-14128

Thanks for reading.

How to find if Oracle Procedure/Table/Index is used?

You are given the task of cleaning up your database like drop unused tables, indexes and   procedures.

If you are running Oracle10g and above, consider yourself lucky; otherwise there is no easy way to get this done.

Even this approach has some pitfalls but should be good enough for most requirements. The approach is limited by the policies you set for AWR repository.

Query DBA_HIST_SQLTEXT where COMMAND_TYPE=47 will list all PL/SQL executions. Also you can get description of numbers in the “COMMAND_TYPE” column by querying AUDIT_ACTIONS table.

Query DBA_HIST_SQL_PLAN for OBJECT_NAME to get tables and indexes that are being used

Note: Oracle 10g/11g requires additional licensing to be purchased for using AWR.

Please note that all the methods describe are not  100% reliable depending on whether the data is captured in DBA_HIST  or whether  your optimizer is picking the correct  index or Oracle using indexes for internal purpose like indexes on foreign key to maintain relational integrity constraints.   So test , test and validate every approach as every application is different

thanks

Oracle Patch sets , PSU and CPU

If you are new to Oracle or lost in 1000s of  pages  of Oracle documentation, then these terms patch set , PSU or CPU is sure to create some kind of confusion.

Based on my experiences with 11gR2, looks like PSU is the way to go.   For example  we  applied PSU3 to resolve mutex issues since there was no one-off fix  w/o  the PSU route.  I also thought that PSUs were more flexible than I initially assumed because  when we encountered another bug(10190759), the bug fix was available only for  11.2.0.2  (w/o PSU 3).  Oracle still recommended applying this patch to 11.2.0.2.3(with PSU3)  and it worked.

Here is brief  summary of what is the difference between patch set , PSU and CPU. I will start with a image, might not be 100% accurate but will help a long way in understanding the differences.

Patch sets and PSU

PSU’s(Patch set updates) are patch sets but with some major differences with respect to regular patch sets.

PSUs are generally low risk and do not contain fixes to bugs that require configuration changes or impact database components like optimizer. To achieve this low risk behavior, Each PSU is limited from 25 to 100 new bug fixes. PSU’s are also well tested by Oracle compared to one off patches.

PSUs are referenced by their 5th place in the Oracle version numbers which makes it easier to track ( (e.g. 10.2.0.3.1) and will not change the version of oracle binaries (like sqlplus, exp/imp etc.) As of now (10.2, 11,2) , the best way to  determine the PSU version installed  is to use opatch utility using the command below

opatch lsinv -bugs_fixed | grep -i  PSU

From the database , you can execute the following query(Assuming that catbundle.sql  was executed when PSU was installed). The last 2 columns will provide information about the PSU.

select substr(action_time,1,30) action_time, substr(id,1,10) id, substr(action,1,10) action,substr(version,1,8) version,
substr(BUNDLE_SERIES,1,6) bundle, substr(comments,1,20) comments from registry$history;

In summary PSU’s  contain fix for bugs that cause

  • Instance crash
  • Wrong results
  • Data Corruption

PSU’s  do not contain fix for bugs  that may result in

  • Dictionary changes
  • Major Algorithm changes
  • Architectural changes
  • Optimizer plan changes

Regular patch sets on other hand include major bug fixes and most of the time requires re-certification. The importance of PSU is diminished once a regular patch set is released for a given version as they tend to be more security related fixes as CPU is part of PSU

PSU and CPU

PSUs  contain CPU’s and are released every quarter (like CPU’s) ; In other words  Critical Patch Update (CPU) is a subset of the Patch Set Update (PSU). CPU’s are built on the base Patch Set version (e.g. 10.2.0.3) whereas PSU are built on the base of the previous PSU (e.g. 10.2.0.3.1)

A PSU can always be applied over any CPU where as applying a CPU over a PSU will roll back the PSU (at least for now). Therefore it is easy to go from CPUs to PSUs and hard to go back to CPUs  from PSUs.

Simple explanation of “cursor: pin S wait on X”

I think it is very  difficult if not impossible to  understand “cursor: pin S wait on X” without the knowledge of “cursor: pin S“.  So this blog will start with “cursor: pin S” along with basics of mutex.

Also see my blog about “cursor:pin s”  with more detailed info https://swervedba.wordpress.com/2011/03/30/cursor-pin-s/

Oracle introduced Mutexes in Oracle Version 10.2 and they stand for mutual exclusion (functionality is similar to mutexes in c).  Mutexes are used to protect data or other resources from concurrent access.

Working of Oracle Mutexes

Oracle uses counters  in implementing mutexes.  The counters are called as reference counters

1. Every time a particular mutex is obtained, Oracle will increment its value.

2. Every time a particular mutex is released, Oracle will decrement its value.

Cursor Pin S : A wait even that occurs when a session wants a specific mutex in shared mode on a specific cursor. As mentioned in the previous section,  Oracle has to update the ref counters  to get the mutex

However it is very important to understand that access to these counters are not concurrent. If there are concurrent sessions trying to obtain a mutex, then only one session can actually increment or decrement the reference counter at a time. Therefore concurrent sessions requesting the mutex must wait. This wait is called as “cursor: pin s” wait.

In short

“cursor: pin s” wait event occurs when Oracle wants a specific mutex in shared  mode on a specific cursor but could not obtain it as it was not able to update the reference counters (Not because another session was holding  the mutex in exclusive mode) .

“cursor: pin S wait on X” wait event occurs when Oracle is not able to get the mutex  in share(S)  mode  because some other session is  holding the  same mutex in  exclusive(X) mode.