The Business of Business is Business

In Bugs, Database tuning, DBA, Grid Control, Oracle database, RAC on July 11, 2011 at 21:58

Winston Churchill said: “Some regard private enterprise as if it were a predatory tiger to be shot. Others look upon it as a cow that they can milk. Only a handful see it for what it really is – the strong horse that pulls the whole cart.”

Have a glimpse at Nasdaq’s article: Upside to Oracle’s Dominant Position in Database Software to double check that namely Oracle is the strong horse. “Oracle continues to maintain its leadership position in the database software market with a share of close to 50%, thanks to constant innovation in its database business and positive feedback from the Exadata line of servers.”

According to Gartner’s market share numbers by relational database management systems, Oracle leads worldwide RDBMS software market share and total software revenue, with its Linux and Unix garnering 74.3% and 60.7% market share respectively.

But for good horses, you need good bug control:

Let me show you an example of a perfectly well working RAC database where all of a sudden you see this “Other” activity in Grid Control:

Drilling down in EM, you witness the unorthodox event “latch: ges resource hash list”. Note that GES resources (GES = Global Enqueue Service) are accessed via a hash array where each resource is protected by a ges resource hash list child latch.

So, what have we done wrong that we have the privilege of witnessing this horrible histogram below (doesn’t the one above look like the top of the head of Bart Simpson)?

Turns out it is Bug 11690639 – High enqueue activity results in “latch: ges resource hash list” waits (Doc ID 11690639.8). Question: is the document ID named after the bug number or is it vice versa?

This is just an example. Many DBAs are nowadays mildly frustrated with the daily routine of bug fixing, looking for workarounds and possible patches. While some years ago the first thing a DBA would do for a problem was to try and tune the database, improve the SQL, etc., now most DBAs try to overrule at first the possibility of a bug.

Bugs are fixed but they re-appear. Fixing a bug can bring another one. Bugs bring systems down. You have covered the infinite availability SLA with RAC, Gloden Gate, Active Dataguard, etc. but one small bug can cause a so severe performance problem that all your efforts and diagnostic tools count for nothing.

How many Database experts talk about bugs at database events? How often do you see blog articles tagged with the word bug? Have you ever tried to look on the Internet for books on database bugs? Check out Martin Decker’s! I enjoy the site a lot!

Oracle keep all their bugs in a database called BugDB. It is a mission critical one and was the first internal application in Oracle to be upgraded with EBR. Check this one out: Online Application Upgrade of Oracle’s Bug DB with EBR!

A Bug DBA is something one would probably not like to be called (unlike a performance tuning DBA) but this phenomenon in the database world is significantly growing its importance and the faster the DBA spots out the bug the less the downtime will be. Unlike using the traditional way where anyone can open an SR with MOS/Metalink and wait for a (possible) resolution.

Bug spotting is of extreme importance nowadays as hours and days of slow performance or downtime can be devastating for the enterprise. And the work of the Bug Buster does not finish after the identification of the bug: then (s)he must decide between recommending patching or using a workaround. And all over again..

  1. Your ideas and Website are awesome but having 12 years of DBA experience with SQL server, Sybase and Oracle I can only say that compared to SQL Server and Sybase. Oracle requires a platoon of DBA’s compared to Sybase/SQL server. I spend more time administering my 12 Oracle databases than I do with my combined 200 Sybase/SQL server databases. From file systems getting filled up with non-sense trace files and other Non readable human language files to constantly troubleshooting OEM… It’s as if Oracle Engineers stand around trying to out complicate the others…. It takes numerous command line task to do what Sybase/SQL server do with one command. Someone mention that “Hey., But look at all the options Oracle provides” And I responded, try troubleshooting all these options… Oracle certainly didn’t embrace the “KISS” concept but took, “Keep it as Complicated as Possible” concept to a higher level.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: