Dontcheff

Archive for the ‘Cloud’ Category

Exadata’s Total Cost of Ownership

In Cloud, Consolidation, DBA, Exadata, Oracle database on November 29, 2012 at 11:55

According to Elbert Hubbard “One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man”.

Of course, neither Exadata nor Bloom filters existed when the Bloomington born philosopher made the above statement.

Forbes magazine published an interesting article this month: Oracle’s Secret Sauce: Why Exadata Is Rocking the Tech Industry.

The article says: “The new approach is embodied in a technology strategy pioneered by Oracle and recently endorsed/followed by IBM (although IBM’s effort to date is rather modest): building richly integrated and fully optimized systems from the ground up, with hardware and software expressly created to work together to deliver maximum performance.”

And as you might guess from the image above, this time I am not only after the technical benefits and advantages of Exadata. I would like to clarify what they bring to business. And see how Oracle Exadata compares to IBM P-Series.

The FactPoint Group created a 30 page cost comparison report for business decision makers: Oracle Exadata Database Machine vs. IBM Power Systems.

In brief, the results of the report are:

• IBM 3 year TCO is 31% higher than Oracle.
• Exadata can be deployed more quickly and easily requiring 59% fewer man-hours than a traditional IBM Power Systems solution.
• Exadata requires 40% fewer sysadmin hours to maintain and operate annually, including quicker support calls because of less finger-pointing and faster service with a single vendor.
• Exadata delivers dramatically higher performance typically up to 12x improvement, as described by customers, over their prior solution.
• Will become even easier to operate over time as users become more proficient and organize around the benefits of integrated infrastructure.
• Supplies a highly available, highly scalable and robust solution that results in reserve capacity that make Exadata easier for IT to operate because IT administrators can manage proactively, not reactively.

Overall, Exadata operations and maintenance keep IT administrators from “living on the edge.” And it’s pre-engineered for long-term growth.

Check Kerry Osborne’s Oracle Blog for more details about the Exadata vs. IBM P-Series comparison.

I personally think that the benefits of Exadata are even much bigger granted the system is properly configured which I see is not always the case but as I said I will not comment on technical issues this time.

But after all, this is a DBA blog, so this part of the research might be of interest for most DBAs:

“For this emerging Database Machine Administrator (DMA) job category, IT employees are cross-trained to handle tasks currently undertaken by admin specialists in hardware, operating systems, network, applications or storage. IT managers who pursue this adaptive path likely will gain operational efficiencies for managing packaged solutions, although it may take several years as IT administrators are trained in new competencies.

The emergence of the DMA also may help restructure IT departments into more efficient operations, but the full benefits of this development cannot be fully realized until most older systems that demand a stove-piped IT organization are decommissioned and IT organizations adapt. At that time, IT operations managers may be able to reduce headcount. In time, packaged solutions should involve not only fewer workers but also fewer IT groups, which should reduce costs; in the meantime IT will be able to do more without adding headcount.”

That is very important! Let me quote here Paul Vallee, who in a recent discussion predicted that in the near future organizations will need few but very skillful DBAs, an opinion I 100% agree with!

“This change in job roles is not necessarily comfortable for everyone in IT because Exadata marginalizes various system administrators as it empowers the DBA: “The DBAs are doing more hardware tasks and diagnostics because most of the Exadata stuff is geared around database commands, not hardware commands or operating system commands. The gearheads have designed Exadata from the DBA’s perspective—when you look at the sys admin portion, it’s all written by a DBA, not by a Sys Admin,” lamented a System Administrator at a Business Services Co.

Other System Administrators have expressed similar sentiments as many of their traditional responsibilities shift towards the DBA—the source of the much of the operational savings we have identified.”

More on the DMA subject from Arup Nanda: Who Manages the Exadata Machine?

For all DBAs: here is an excellent book on Exadata: Expert Oracle Exadata, by Kerry Osborne, Randy Johnson and Tanel Põder.

Curb Your Enthusiasm: Database Virualization

In Cloud, Consolidation, DBA, Oracle database on September 29, 2012 at 00:38

Larry David says: “You write about what you know”.

RAC is one of the most controversial topics among database experts.

An extremely interesting article, called Database Virtualisation: The End of Oracle RAC? was published this month (September 2012) on FlashDBA.com.

I recommend it to every Database Architect, DBA and Developer! Along with the comments after it. Plus its links to related papers and blogs.

Another interesting post on the RAC issue, entitled To RAC or not to RAC and its sequel To RAC or not to RAC (reprise part 2) is worth reading as well.

To make it simple, let me quote techtarget.com:

“The biggest disadvantage is that you are adding more complexity to your database architecture. With more complexity comes a higher cost in maintaining and administering the database along with a higher chance that something will go wrong.

The second biggest disadvantage is the cost associated with RAC. Oracle is touting RAC on Linux as a way to acheive cost savings over large Unix servers. With RAC, the costs shift from hardware to software as you need additional Oracle license fees. The big question is will this shifting of costs result in any cost savings. In some cases, yes, and in other cases, no.”

Which would be the third big disadvantage of RAC? I say the bugs! Or let me put it more mildly: RAC just develops random features. And hunting errors in RAC is complex, right? On top of ORA-600, we now have even ORA-700.

Let me offer you some quotes from the RAC debate on FlashDBA.com:

  • “Then there are the younger DBAs looking to gain more experience, who may say that RAC is a great thing. Secretly that might not necessarily be true but they want the experience.”
  • “We also see that there are “no application changes necessary”. I have serious doubts about that last statement, as it appears to contradict evidence from countless independent Oracle experts.”
  • “Complexity is the enemy of high availability – and RAC, no matter how you look at it, adds complexity over a single-instance implementation of Oracle.”
  • “At no time do I ever remember visiting a customer who had implemented the various Transparent Application Failover (TAF) policies and Fast Application Notification (FAN) mechanisms.”

  • So, is Database Virtualization the answer?

    I think that database virtualization is a concept that has been misunderstood and most of all wrongly defined by many. According to the Oxford Dictionarries, in Computing, virtual means something which is not physically existing as such but made by software to appear to do so.

    Decide for yourself what is then a virtual database. At least, it is not a database built in a virtual server! I did discuss that at the Oracle ACE Director Product Briefing at Oracle headquarters this week with world’s top database experts and what can I say: the topic is highly controversial.

    Let us go to the Oracle Database Documentation Library and search for virtualization. Here is the result:

    And you will get the same result for virtualisation (with “s”).

    So, what do we get? Not much.But let’s continue reading:

  • Database Virtualization Part 2 – Flash Makes The Difference
  • The Do’s And Don’ts Of Virtualizing Database Servers
  • Disadvantages of Virtualization, What’s Your Opinion?
  • Virtualization Best Practices
  • Let us wait for next Oracle’s database release and see what it will offer us.

    Exadata Consolidation: “You must spend money to make money”

    In Cloud, Database tuning, DBA, Exadata, Oracle database on July 20, 2012 at 20:28

    Titus Maccius Plautus, a roman poet and philosopher who lived from 254 BC to 184 BC, said: “Wisdom is not attained by years, but by ability”. And also the famous quote in the title above.

    What do I have in mind? Even midsize companies have nowadays dozens, if not hundreds, of database instances. Corporate IT departments would not surprise anyone if they support a couple of thousand databases.

    The consolidation of these databases can be achieved in several ways. But here are some general guidelines on how to accomplish this task by using Exadata in the most optimal way in order to maximize cost reduction, high availability, secure separation of administrative duties and ease of performance tuning, management and monitoring.

    It is really surprising how slowly companies adopt database consolidation granted the pressure IT management has in every possible direction. We can speculate for hours why so, but what I will concentrate on, are the technical aspects of Exadata consolidation.

    Oracle recommends the creation of Exadata Hardware Pools. Hardware Pool is a machine or group of machines used as the target consolidation platform.

    According to Oracle’s white paper “Best Practices For Database Consolidation On Exadata Database Machine“, an enterprise might create multiple Hardware Pools to make each consolidation target platform more manageable. The recommended minimum Hardware Pool size Oracle Maximum Availability Architecture Exadata Consolidation Best Practices is Exadata X2-2 Half Rack and the maximum recommended Hardware Pool size is two Exadata Database Machines Full Racks (plus additional Exadata storage expansion racks if required). Hardware Pools that fall within this range are the most common Exadata configurations for consolidation and provide sufficient capacity to efficiently achieve objectives for database consolidation.

    The recommended storage configuration is one shared Exadata storage grid for each Hardware Pool. This storage grid contains all Exadata cells and Exadata disks, and is configured with either ASM high or normal redundancy. The recommended setup for Oracle Grid Infrastructure (which includes Oracle Clusterware and Oracle ASM) is to use one cluster per Hardware Pool.

    Oracle has recommended several parameter settings for Exadata database consolidation:

    If PageTables in /proc/meminfo is set to more than 2% of the physical memory size, then set the operating system parameter HugePages to the sum of all shared memory segments. Starting in 11.2.0.2, setting the database initialization parameter USE_LARGE_PAGES=ONLY on all instances prevents any instance from starting unless sufficient HugePages are available. Hugepages can only be used for SGA, so do not over-allocate. Also, the database parameters MEMORY_MAX_TARGET and MEMORY_TARGET are not compatible when HugePages are enabled. This is only for Linux. On Solaris, HugePages are automatically configured and used via intimate shared memory (ISM).

    Operating system setting:

    Set the number of shared memory segments (kernel.shmmni) greater than the number of databases.
    Set the maximum shared memory segment size (kernel.shmmax) to 85% of physical memory size, which is the default.
    Set the maximum total number of system semaphores (SEMMNS) greater than the sum of all database processes.
    Set the maximum number of semaphores in a semaphore set (SEMMSL) greater than the largest number of processes in any single database.

    Exadata Memory:

    1. Exadata X2-2 based on the Sun Fire X4170 Oracle Database Servers (also known as V2) has 72 GB per database server.
    2. Exadata X2-2 has 96 gigabytes (GB) of memory in the default configuration, with an option to expand to 144 GB of memory (with the Exadata memory expansion kit).
    3. Exadata X2-8 has 1 terabyte (TB) (with the X4800) or 2 TB (with the X4800M2) per database server.

    An important rule:

    OLTP applications: SUM of databases (SGA_TARGET + PGA_AGGREGATE_TARGET) + 4 MB * (Maximum PROCESSES) < Physical Memory per Database Node
    DW/BI applications: SUM of databases (SGA_TARGET + 3 * PGA_AGGREGATE_TARGET) < Physical Memory per Database Node

    Remember to enable instance caging!

    For the Oracle ASM instance, set PROCESSES= 50 * MIN ( # database instances on db node+ 1, 11) + 10 * MAX (# database instances on db node – 10, 0).

    Limit PARALLEL_MAX_SERVERS:

    1. X2-2: sum(PARALLEL_MAX_SERVERS) for all instances <= 240
    2- X2-8: sum(PARALLEL_MAX_SERVERS) for all instances RECOVER MANAGED STANDBY DATABASE … PARALLEL 16.

    Limit the number of processes and connections to the database servers:


    -
    For Exadata running Exadata Storage Server Software 11.2.3.1 or higher, configure a maximum of 60,000 processes per Hardware Pool. Upper limit target is 7,500 processes per node for X2-2. Upper limit target is 30,000 processes per node for X2-8. For Exadata running Exadata Storage Server Software 11.2.2.4.2 or less, configure a maximum of 20,000 processes per Hardware Pool. Upper limit target is 2,500 processes per node for X2-2. Upper limit target is 10,000 processes per node for X2-8.

    The temporary tablespace should be:

    BigFile Tablespace,
    Located in DATA or RECO, whichever one is not HIGH redundancy,
    Sized 32GB initially,
    Configured with AutoExtend on at 4GB,
    Configured with a MaxSize defined to limit out of control growth.

    Answers to several additional Exadata related questions can be found in the following blogs/articles:

    Who Manages the Exadata Machine? by Arup Nanda
    Upgrade Exadata to 11.2.0.3 by Gleb Otochkin from Pythian
    Best Practices For Database Consolidation On Exadata by Javier Puerta
    Consolidation Strategies for Oracle Exadata and Oracle Database Machine by Dan Norris, X Team, Oracle Exadata Development
    Expert Oracle Exadata by Kerry Osborne, Randy Johnson, Tanel Põder
    Oracle Exadata – A platform for consolidation by Umesh Tanna

    An excellent reference is the Consolidation Parameters Reference Table available at MOS: “Oracle Sun Database Machine Setup/Configuration Best Practices” [ID 1274318.1]

    Back to the saying of Plautus: one does not have to wait for years in order to start implementing an Exadata consolidation. The product has the perfect ability to serve its many purposes: this is software, not wine!

    On DBA innovation: who is afraid to fail will keep falling forever

    In Cloud, Database tuning, DBA, Exadata, Grid Control, Oracle database on April 24, 2012 at 19:06

    Managers always ask software engineers to deliver something sooner rather than waiting to deliver everything later.

    How come it is fine to deliver an incomplete low quality IT product just because it is delivered on time? Most of us have been on several occasions limited by that deadline and time-schedule in terms of creativity, innovation and pro-activeness.

    Innovation in database management and database administration has nothing to do with DBAs from whom you often hear phrases like “If it ain’t broke, don’t fix it” or DBAs who follow blindly the KISS principle.

    In DBA terms, innovation is the process of introducing and implementing new features in the database and using new options and database products.

    A good example is the adoption of Oracle Enterprise Manager Grid/Cloud control. I have seen so many excuses for not implementing it or delaying its implementation. And the benefits and savings of OEM implementation can be measures in times!

    Do you wonder how quickly and efficiently one can find out all details of a certain SQL statement without Cloud Control? No other database brand has so highly sophisticated tool for performance analysis like Oracle has. See all the details offered with one mouse click: Exadata Cell Offload Efficiency (96% in this case), Wait Activity in detail, use of the Result Cache, etc. All that from a single screen!

    Accepting new innovative database properties, tools and appliances is hard for many IT architects, IT managers and most of all for DBAs who have the primary responsibility to test, verify and promote these features.

    Let us have a look at one other innovation from Oracle. Implementing Exadata can be strongly considered if any of the following 5 points are in the IT roadmap:

    1. Implementing a new Business Intelligence solution
    2. Datawarehouse licenses are up for renewal
    3. Database platform consolidation exercise
    4. Storage requirements are increasing and the performance is decreasing
    5. Performance of transactional systems requires major improvements

    Look at the list below and think how many of these are used by your company or client:

    - Exadata
    - Enterprise Manager Cloud Control
    - Edition-Based Redefinition
    - Advanced Compression
    - Oracle Database Appliance
    - Automatic SQL Tuning
    - Total Recall
    - Real Application Testing
    - One of these init.ora parameters: db_ultra_safe, result_cache_mode, optimizer_capture_sql_plan_baselines, awr_snapshot_time_offset

    Although I did not put Enterprise Manager Cloud Control on top of the list, is it still a must for every Enterprise using Oracle products. On the light side, I was recently asked if you can see the temperature of a given computer from OEM :-) Here we go:

    Jon Taplin said it very well in his article on Steve Jobs and Innovation: “At the Innovation Lab we try to inculcate the notion that you can’t be afraid. You can’t be afraid to fail. You can’t be afraid to “be different”. You can’t be afraid to celebrate the weird mix of art and science that is true innovation. Steve Jobs embodied all of those qualities. I wrote a bit about him in my new book and there is a cool video in the book of his graduation speech at Stanford that you will see replayed too often in the next few days.”

    As a DBA, one should try to step out of his comfort zone where (s)he is surrounded by the everyday administrative tasks and reactive performance tuning work and try to make difference in the enterprise by acting more like a Database Architect than a Database Operator. Prove the complexity and importance of the DBA role!

    P.S. Often in IT, the way from Insight to Action is longer than the Way of St. James.

    New DBA Features in Oracle Enterprise Manager Cloud Control 12c

    In Cloud, Database tuning, DBA, Grid Control, Oracle database on October 22, 2011 at 10:17

    During Oracle OpenWorld 2011, Oracle introduced Enterprise Manager Cloud Control 12c Release 1 (12.1.0.1). I have already done the first installation on OEL Linux 6 and I must say I am impressed with the product.

    Oracle Enterprise Manager Cloud Control 12c came with new features covered in 12 areas:

    - Framework Enhancements
    - Database Management Features
    - Fusion Middleware Management Features
    - Oracle Fusion Applications Management
    - Application Performance Management Features
    - Cloud Management Features
    - Incident Management Features
    - My Oracle Support Integration
    - Security Features
    - Lifecycle Management and Provisioning Features
    - Extensibility Support
    - Coherence Management Features

    In this post, I will present the new 20 DBA features of OEM CC 12c as documented by Oracle (the screenshots in between are from my own installation):

    1. Database Creation Via Enterprise Manager Cloud Control

    A wizard allows you to create an Oracle Database from within Enterprise Manager Cloud Control. You can create different configurations, including Single Instance and Real Application Clusters (RAC) databases, as well as file system and Automatic Storage Management (ASM) storage options.

    2. Database Upgrade Via Enterprise Manager Cloud Control

    You can now upgrade Single Instance and Real Application Clusters (RAC) Oracle databases through Cloud Control. This feature makes it possible to manage upgrades from a single console. You no longer have to access each individual database machine to perform upgrades.

    3. Database Cloning Enhancements

    Cloning procedures have been enhanced to capture configuration properties as well as the software payload. This is particularly useful when migrating databases from development to test to production or the reverse. A new EMCLI verb (clone_database) allows for database cloning using the same job type as the existing Clone Database feature of Cloud Control.

    4. Oracle Exadata Server Management Enhancements

    Oracle Exadata management capabilities now provide support for full target lifecycle management:

    - Automatically discover Exadata targets
    - Automatically create an Enterprise Manager System for end-to-end monitoring
    - Provide extensive administration capabilities for databases, Exadata cells and Infiniband switches
    - Simplify performance diagnostics with the help of in-depth performance charts covering all Exadata components

    5. Manage Automatic Storage Management (ASM) Clusters as a Target

    You can now manage clustered ASM resources as a single target, instead of each individual ASM instance having to be managed separately. Incident management and metric monitoring can be managed for the entire cluster.

    6. Database Configuration Compliance Standards Support

    Oracle database configuration data can now be managed within the new configuration and compliance standards frameworks.

    7. Emergency Performance

    This feature allows a DBA to diagnose and troubleshoot a hung or extremely slow database using the memory access mode. This mode is designed to bypass the SQL information retrieval layer and read performance statistics directly from the SGA of the target.

    8. Database Backup and Restore Enhancements

    You can now use Enterprise Manager Cloud Control to centrally maintain the settings for Oracle database and file system backups. This enhancement enables you to apply uniform settings to multiple databases and file systems when backing up multiple objects—all in one operation. Users can perform a backup on multiple databases concurrently or sequentially in one deployment procedure. An Oracle Home OSB tape backup can be restored either to the original or to a different location, and the restored Oracle Home can be reconfigured to function in the newly restored location.

    9. Database System Discovery Enhancements

    As the database system is now built upon the new target and association model, you can use it to monitor and manage a database’s storage, connectivity, and high availability. This also enables you to easily monitor and manage the applications that are dependent on the database. The database discovery functionality is enhanced to work with the new discovery framework and to provide a reliable workflow to create a database system.

    10. Change Plans Added to Change Management Pack

    As part of the Oracle Change Management Pack, the new Change Plans function allows application developers and database administrators to encapsulate schema changes needed to be made to a database into a “change plan,” which can be used to document, capture, and apply schema changes. Change Plans are also integrated with developer and DBA tasks into SQL*Developer and Oracle Enterprise Manager task automation. This integration reduces the manual processes between the various stakeholders involved in the process of promoting planned changes across enterprise databases while ensuring the integrity of the process.

    11. Compare Period Advisor

    This feature compares the performance of a database over two different time ranges. It analyzes changes in performance, workload, configuration, and hardware to highlight changes between the two time periods. The Compare Period Advisor gives the DBA the ability to compare two arbitrary periods of time.

    12. Compare Functionality

    The Compare functionality has been enhanced with new capabilities such as template support, system level comparison, and change notification. Users can now selectively include or ignore types of differences. Output of a comparison can easily be saved and exported, both in printable (for example, plain text) and data-centric (for example, CSV) formats. Users can select comparison start and end dates and view a history of changes for composite targets.

    13. Active Reports

    A new Active Reports function allows users to save performance data into an HTML file. Once saved, the report can be used for offline analysis or sent to other users, including Oracle Support. Active Reports enhances the visual representation of performance data and facilitates the convenient exchange of complex data.

    14. Real Application Testing and Data Masking Integration

    Real Application Testing and Data Masking integration provides users with the ability to perform secure testing in situations where data in production needs to be shared by nonproduction users due to organization or business requirements. Typically testing is done in a nonproduction environment or by a different group or organization. This integration addresses a common requirement that the data used for testing be shared in a manner that adheres to data privacy and compliance regulations.

    15. Application Templates for Data Masking and Data Subsetting

    This feature provides predefined data masking and data subsetting templates for applications. It allows users to automatically create test systems based on best practices recommendations.

    16. Data Subsetting

    Data subsetting provides the ability to create a smaller sized copy of the original production data that can be given to developers for testing. While it is a data subset, the referential relationships are preserved so that the data set is complete. This allows enterprises to lower storage costs while making production data available to developers for testing, without having to incur the storage footprint of the entire production database.

    17. Application Data Model Support for Data Masking

    The application data model (ADM) now stores the sensitive data elements used to generate mask definitions dynamically. Instead of having to manually discover sensitive data, the application data model identifies and stores the sensitive data elements.

    18. Reversible Data Masking

    Using encryption and decryption algorithms, reversible masking allows encryption of a user’s data deterministically into a format chosen by the user as a regular expression. Unmasking reverses the process to revert back to the original data. This feature is useful in environments where sensitive data needs to be masked and sent to a third party for processing. Coupling integrated masking with the application data model (ADM), an application’s data model is now available for certain packaged applications and can serve as a knowledge base containing sensitive column and data relationships.

    19. Performance Diagnostics Enhancements

    With the interactive user interface in the Active Session History (ASH) Viewer, users now can visualize the many performance dimensions that were not available to them in earlier releases. The Enhanced Enterprise Manager Performance and Top Activity pages allow users to visualize the multidimensional data in ASH. The ASH viewer enhances the performance troubleshooting capabilities of a DBA by providing the facility to detect skews in workload. Emergency ADDM adds performance diagnostics for databases suffering from severe performance problems.

    20. Streams and XStreams Support

    Streams and XStreams configurations can now be managed and monitored using Cloud Control. In addition to improvements in configuration and performance monitoring screens, logical change record (LCR) tracking is available for high-level diagnosis of replication issues. Cloud Control also simplifies the management and monitoring of replicated environments.

    Oracle Enterprise Manager Cloud Control 12c

    In Cloud, DBA, Grid Control, Oracle database on October 4, 2011 at 00:57

    It is now official: Oracle announced on October 3rd the new version of Enterprise Manager Cloud Control 12c Release 1 (12.1.0.1).


    This is the first solution in IT to combine management of the full Oracle stack with complete enterprise cloud lifecycle management.

    You may download it only for Linux x86-64 (64-bit).

    The Enterprise Manager Cloud Control Documentation gives you the following:

    Basic Installation Guide
    Advanced Installation and Configuration Guide
    Upgrade Guide
    Administrator’s Guide
    Cloud Administration Guide
    Licensing Information

    There are 4 mandatory plug-ins that are installed by default with Enterprise Manager Cloud Control:

    Oracle Database plug-in
    Oracle Fusion Middleware plug-in
    Oracle My Oracle Support plug-in
    Oracle Exadata plug-in

    Enterprise Manager offers the following management packs for Oracle Database:

    Diagnostics Pack for Oracle Database
    Tuning Pack for Oracle Database
    Database Lifecycle Management Pack for Oracle Database
    Configuration Management Pack for Oracle Database
    Provisioning and Patch Automation Pack for Oracle Database
    Change Management Pack for Oracle Database
    Oracle Data Masking Pack for Oracle and non-Oracle Databases
    Oracle Test Data Management Pack for Oracle and non-Oracle Databases

    Top 3 new features for DBAs:

    1. Real-time ADDM. You can connect to the database and investigate what is going on when you cannot connect to the database because it is hanging on it is extremely slow. This is for real, connection to the database is in diagnostic mode, there is direct connections to the SGA.

    2. Active Session History (ASH) Analytics. It is the adavnced version of Top Activity.

    This is a snapshot from load map view mode:

    3. Compare period ADDM. You can identify why during a certain period activity was slow when no changes to the SQL statements were made. Here is a snapshot from ADDM compare:

    Great product from Oracle and this is not any longer just a DBA tool.

    From yesterday’s Total Cloud Control session: Richard Sarwal and Sushil Kumar:

    More to follow on DBA new features in “New DBA Features in Oracle Enterprise Manager Cloud Control 12c”.

    Follow

    Get every new post delivered to your Inbox.

    Join 115 other followers