Oracle Enterprise Manager Administrator's Guide Release 2.2 Part Number A85248-01 |
|
The Event system allows you to monitor your network for specific conditions, such as loss of service or lack of storage, that may occur in your managed environment. You select tests to run on managed targets (databases, nodes, listeners, or other services), then set the threshold parameters for which you want to be notified. You can share events with other administrators, in addition to being able to notify specific administrators when an event condition occurs. For some event tests, you can also choose to execute a fixit job that automatically corrects the problem.
The following topics are discussed in this chapter:
This section discusses the enhancements to the Events System in Enterprise Manager 2.2.
These event tests monitor operating system-specific metrics for Solaris, HP-UX, Compaq Tru64, IBM AIX and Windows NT. These new events require an 8.1.7 or higher Intelligent Agent. As with other Advanced Events, these events are available with the Oracle Diagnostics Pack.
Duplicate events can now be registered against 8.1.7 or higher Agents. Duplicate events are those that contain the same event tests and parameters against the same target. For example, 2 or more administrators can create their own events that register the same Tablespace Full event with the same parameters against the same tablespace.
You cannot, however, have duplicate event tests within the same Event. For example, if you create an Event called "Database Availability", it cannot have 2 or more of the same Database Up/Down event test against the same database A.
Refer to the Enterprise Manager release notes for details on duplicate events with earlier versions of Enterprise Manager and the agents.
The Node Up/Down event test is enhanced to distinguish between situations where the target node is unreachable, the agent is down, or the agent is not responding.
For Events that monitor targets with multiple subcomponents, (e.g. a group of tablespaces), notifications will now be sent each time the status of any subcomponent changes, even if it does not change the overall severity of the Event.
For example, you register a Tablespace Full event for tablespaces A, B, C. If tablespace A becomes full, the event triggers an Alert. If tablespace B also becomes full, then even though the event is still in Alert due to tablespace A, another notification will be sent due to tablespace B becoming full.
When there are problems evaluating an event (e.g. an archive full event against a database that is NOT running in archivelog mode), then an alert may be sent indicating the event is in error. This error state is denoted by a yellow hexagon. In most cases, events in error are due to user errors and can thus be corrected.
Some examples of error situations are:
Refer to "Interpreting Events" on page 5-8 for more information.
When registered against 8.1.7 or higher Agents, the Datablock Corruption event test will now also monitor for ORA-1157 (cannot identify/lock data file string - see DBWR trace file) and ORA-27048 (skgfifi: file header information is invalid) errors.
The User-Defined SQL event test can support pl/sql functions. This means you can select from a pl/sql function and have the return value evaluated against a specified threshold. The actual pl/sql function needs to be defined separately.
To simplify setup and configuration, the Event Handler is now solely available as an external service of the Management Server. Error handling has also been improved.
When creating/editing events, this feature will allow users to select an individual event test and press the F1 key or click the Help button to access online help for that particular event.
The Event system allows you to efficiently monitor a large system. Using the Event system and Intelligent Agents, you can effectively monitor any number of databases, nodes, or other services 24 hours a day, and be alerted when a problem or specific condition is detected. You can also pinpoint only the services you wish to monitor. The Event system can be extended to include other third-party applications that detect events independent of the Intelligent Agents.
In the Event system, event settings are stored based on the administrator registering the event. This allows administrators of large systems to customize their event systems to their preferences and tasks. Administrators receive messages for events for which they have been selected to receive notifications by other administrators.
The Event system includes the following processes:
You need to create and register events, which are simply a group of event tests that you want to run on your managed systems. Oracle Enterprise Manager includes a variety of predefined event tests that you can use when creating events. The event tests are grouped by service type, for instance:
You can create events using the predefined event tests that have been installed with Oracle Enterprise Manager. See "Event Categories and Types" on page 5-10 for more information.
The events are created with information entered in the Event property sheet. You determine parameters such as the destination that is monitored, the specific tests to perform, the frequency that the event test is executed, and whether other administrators can share the events. See "Permissions" on page 1-23 for more information. Some event tests have parameters with threshold values that you can customize for your system. See "Event Parameters Page" on page 5-34 for more information. To use the Event system, an administrator must have sufficient privileges to access database objects from the Console. Under most circumstances, full DBA privileges are not required, nor would be appropriate to assign full DBA privileges to every administrator. For this reason, the Event Handler role was created.
Beginning with Oracle 8.0.6 databases and higher, the OEM_MONITOR role is created by the Oracle database creation scripts. This role permits access to database functionality within Enterprise Manager. For example, running events against a database (tablespace full, buffer cache hit ratio) or browsing through the objects in a database via the Console Navigator tree. These types of functionality require database credentials on which to perform these operations. Rather than granting the powerful DBA role to the database credentials, many administrators prefer to provide only the necessary privileges required to do these operations. Granting the OEM_MONITOR role to the database credentials, ensures that the user has the minimum sufficient privileges required for these operations.
For database users on 7.3.x databases, you need to define the OEM_MONITOR role manually.
Here are the steps you need to perform:
drop role OEM_MONITOR; create role OEM_MONTOR:
grant connect to OEM_MONITOR;
grant analyze any to OEM_MONITOR; grant create table to OEM_MONITOR;
grant select_catalog_role to OEM_MONITOR;
You are now ready to grant the OEM_MONITOR role to the database user that will be used as "database preferred credentials" in Enterprise Manager. In addition to granting the OEM_MONITOR role to a user, you must also ensure that the QUOTA for the user account is set to UNLIMITED.
The "Continued Row" event test needs to analyze results into a table so it needs both the "analyze any" and "create table" privileges.
Events are registered, or submitted, to specific destinations, such as nodes, listeners, or databases. The status of a registered event is viewed in the Registered page of the Event pane.
The event scripts are executed on nodes with the permissions of the Intelligent Agent. However, some of the database event tests, such as Continued Rows, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Preferred Credentials" on page 1-25 for more information.
The Intelligent Agent is responsible for detecting when a specific event condition has occurred. The Intelligent Agent first notifies a Management Server, which in turn notifies interested administrators either through the Oracle Enterprise Manger Console, or by external means such as Email or Paging.
The Management Server is responsible for registering event information with the appropriate Intelligent Agents on nodes in the network. You determine the frequency that an Intelligent Agent checks an event. See "Frequency" on page 5-32 for details on setting the frequency interval for an event. An exception to this is the Up/Down (node) event test, which is checked at an interval set by the system itself. See "Fault Management Event Tests" on page 5-11 for more information on this event test.
When an alert condition occurs, the Intelligent Agent is responsible for notifying the Management Server. Each event is logged in the repository and can be viewed and acknowledged in the Alerts page of the Console. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.
Events can consist of multiple event tests. If any one of these tests identify a specified condition, the event is triggered and a notification is sent to the Console. If enhanced notification is configured for your system, paging and/or email notifications are sent.
Event notification occurs as follows:
Enterprise Manager administrators can be notified in various ways, such as electronic mail or paging, depending on the administrator's setups and permissions. You need to set up the notification services and determine the administrators that need to be notified for the events. See "Event Permissions Page" on page 5-35 and "Permissions" on page 1-23 to determine the administrators that receive notifications. See "Notification" on page 1-12 and "Schedule" on page 1-22 to determine how and when an administrator is notified.
If you plan to notify administrators with email or paging, you need to make sure the following are set up properly:
An event is composed of one or more event tests. While an individual event test may result in a different status (For example, some clear, some are in alert), there is a general status for the Event. To determine the general severity for the event, the following rules apply in succession:
You can still see the individual status of each event test in the Event Viewer.
All events return values and some events produce output messages. The events return different icons depending on the severity of the event. These severity levels are determined by parameter thresholds you set for the event tests during event creation. The colors are displayed on the event severity icon that is located:
The colors of the event severity icons are:
An error state indicates there is a problem with the evaluation of the event condition, as opposed to a threshold being met. Examples of error states are: registering an Archive Full event against a database in non-archivelog mode, registering an event that monitors segments but specifying a filter that excludes all available segments.
A gray flag represents an "unknown" state where it is not possible for Enterprise Manager to ascertain the event status because the node is unreachable or the Intelligent Agent is not working. The gray flag will appear on the group pane and as the flag for the event in the Alert tab if your event includes at least one up/down event test (any target: node/database/listener). When the gray flag occurs, it will be set for the Event. When you see the event in the Event viewer, the flag for the up/down event test will be gray, and the flags for the other event tests will remain the color of their original state.
If you have a single db up/down event test in an event, it will also turn gray (at the Event level). When you look at the Event in the Event Viewer, you will see two event tests: db up/down and node up/down (which is always added whenever you have any up/down event test). Both these event tests will have gray flags.
If you have an event that does not include any up/down event test, then it will remain the same color even though other events are showing gray flags.
If an UpDown event test is specified for a particular event, and the node to which the event is assigned goes down, the event will return an "unknown" severity (gray flag). The original event severity will be restored when the node comes back up. At least one UpDown event test (node, listener, database, etc.) must be specified for an event in order for the "unknown" severity to be returned. Events not containing an UpDown event test will not provide any node down notification.
Some events, such as Probe and User Blocks events, do not return a warning value because the warning threshold parameter is not used. The event has either occurred or not occurred.
When an event occurs, you need to correct the problem. In some cases, you can create a fixit job that responds to specific event conditions. These situations are noted in the online help for Oracle events.
In other cases, the solution requires careful attention by a system administrator. For example, space management event conditions may require an administrator to increase space requirements and resource management conditions may require an administrator to adjust initialization parameters.
For information on how to correct event conditions, refer to the appropriate documentation. To correct Oracle database problems, refer to the Oracle Server Administrator's, Tuning, and Reference Guides. For network problems, refer to the Oracle networking guides for your system.
The Oracle event tests for the database, listener, and node destination types are grouped into categories:
Only the UpDown event tests are included with Oracle Enterprise Manager. These fall under the category of `Fault Management event tests'. Additional advanced events for all categories are available with the optional Oracle Diagnostics Pack. Beginning with the Oracle Diagnostics Pack for Enterprise Manager version 2.2, operating system-specific tests are also available for NT and UNIX platforms.
See the online help for Oracle predefined event tests, "Oracle Event Tests" on page 5-40, and the Diagnostic Pack documentation for information on events and their parameters. All the Node events are supported on Unix and Windows NT platforms. For other platforms, see your platform-specific documentation.
This category of event tests monitors for catastrophic conditions on the system, such as a database, node, or listener is down. Immediate action must be taken by the administrator. Examples of event tests available in this category include:
Most of the fault management event tests do not require any threshold values because the event test only checks whether the service is up or down. For the Alert event test, the event test checks whether error messages are written into the database alert log file.
The UpDown event tests are provided with the Enterprise Manager base product. These event tests check whether a database, listener, or node is available. With the UpDown event test for databases or listeners, you can use the Startup Database or Startup Listener task as a fixit job to re-start the database or listener. To avoid executing that job when the database or listener is brought down intentionally, you need to remove the event registration.
This category of event tests track possible space problems, such as running out of space on a disk or archive device. Examples of space management event tests in this category include:
To check for space management events, set a threshold on the free space left. For example, set an alert if the free space on a disk falls below a specific number of bytes. In order to properly choose the threshold value, you need to know the characteristics of the tablespaces. For example, you would want to know whether the tablespaces contain online transaction processing (OLTP) tables or decision support tables. The former usually has a very fast growth rate, while the latter almost never grows.
This category of event tests track possible resource problems, such as exceeding datafile or lock limits. Examples of resource management event tests in this category include:
To check for resource management events, set a threshold on the percentage of a resource used. For example, you can set an alert if the percentage of the datafile resource used is greater than a specified value.
This category of event test monitors the system for performance problems, such as excessive disk input/output or library cache miss rate. Examples of events in this category include:
To check for performance management events, set a threshold on a system value. For example, you can set an alert if the library cache miss rate is greater than a specific value. The set of threshold values is system specific, depending on the hardware platform, number of users, and other factors.
Unsolicited event tests are event tests that have been initiated outside the Enterprise Manager Event system. An event is considered unsolicited if it is raised by a process other than the Oracle Intelligent Agent, but is running on the same node as the Intelligent Agent. These events are usually provided by third-party software. Creating an unsolicited event allows you to integrate and monitor third-party events. Essentially, there are two phases to setting up an unsolicited event:
To register interest in an unsolicited event, check the Unsolicited Event Test box in the Event General page and complete the information in the succeeding property sheet pages. Select the Unsolicited event in the Test page and complete the Parameters pages. Information on how to fill out the Parameters page is discussed in the next section. After completing the unsolicited event, you can save and submit the event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.
Because unsolicited events originate outside the Event system, you may wish to screen only for specific external events. The Parameters page for the Unsolicited Event Test allows you to filter unsolicited events based upon the following field entries:
This is the four-part name of the event in the form:
/vendor/product/category/name
You can enter any character strings but all four parts and the forward slashes (/) are required. The eventname is assumed to be in 7-bit ASCII, so that it never changes regardless of platform or language. The name of the event that fires must match the value specified in this parameter field in order for the unsolicited event to fire.
Enter a "*" if you do not want to filter on an event name.
An object can be any service the unsolicited event is monitoring, such as a database, listener, or node name. Important: The object must be a service that the Intelligent Agent has already discovered.
Enter a wildcard character ( "*") if you want to receive unsolicted events from any type of object. Or, you can use the percent sign "%" for partial filtering. For example, ORCL% for events fired on objects whose name starts with "ORCL."
Enter the numeric severity level (-1, 1, 2) corresponding to the severity of the event.
Severity levels map to different flags in the Console Event Pane:
Flag Color | Severity | Severity Level |
---|---|---|
Green |
Cleared |
Severity -1 |
Yellow |
Warning |
Severity 1 |
Red |
Critical |
Severity 2 |
Message is a quoted text string that is displayed in the Console, such as "File not found." If a message filter is specified, then the event is passed only if the message matches what is specified in the message filter.
You can specify wildcard characters as part of the filter i.e. Message Filter : ERROR%. In this case the unsolicited event should fire on anything with a message that starts with "ERROR:"
To raise unsolicited events, users have a choice of a command-line interface (oemevent executable) or an OraTcl verb (orareportevent). The related syntax is as follows:
oemevent [event_name] [object_name] [severity] [message] orareportevent [event_name] [object_name] [severity] [message]
where event_name is the name of the event that triggered the unsolicited event. object_name is the name of the object that the event is monitoring, severity is the level of severity for the event, and message is the text string displayed in the Enterprise Manager Console. For additional details, please refer to the Intelligent Agent User's Guide.
Note that the severity is specified as a string in oemevent and as an integer in orareportevent. Also, note that the event_name must be a four-part string of the form /a/b/c/d, where the different elements may be used to organize the event test within a hierarchy of event tests. For example, /myevents/node/files/filefound may be an event test you developed. It relates to nodes, more specifically space on nodes, and it monitors for the existence of a particular file.
For an example of using OraTcl to trigger a third-party event, see Tcl Script Examples in the Run Tcl job task. For information on OraTcl and event scripts, see the Oracle Intelligent Agent User's Guide.
Typically, unsolicited events are evaluated and raised by third-party software. Enterprise Manager allows you to implement unsolicted events through the Job system and Tcl. You create a Tcl job and submit it as a periodic job. The Tcl contains logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Since the job is submitted as a periodic job, the underlying test is evaluated periodically like all regular Enterprise Manager event tests. Using techniques like the ones scribed below, users of Enterprise can implement event monitoring specific to their environments.
It is possible to submit a job with an imbedded OS command task that executes the oemevent program and passes the program all necessary arguments. All users that have registered for the event raised by oemevent, will receive the event notification. The event has to be known to the administrator submitting the job that raises it.
However, the job that raises the event may contain enough logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Such a job may be submitted as a periodic job so that the underlying test is evaluated periodically, which is similar to regular OEM event tests.
Unsolicited events (their logic actually) are evaluated in their own process and within the proper OS security envelope. For this reason, no longer pose security or robustness threats to the system. What we have introduced here is a procedure where one needs to submit a job in order to monitor for an event.
Consider as an example how to implement an event test which triggers when a particular file is found. Lets call this event /myevents/node/files/filefound. The following Tcl script needs to be submitted as the job:
# event name set event_name /myevents/node/files/filefound # filename to look for comes at the first (and only) argument set file_name [lindex $argv 0] # check for the file, and if it's found trigger the event as critical if { [file exists $file_name] } { orareportevent $event_name $oramsg(oraobject) 2 "$file_name found" }
In order to receive this event, a user needs to register an event with the Unsolicited test selected and configured to filter in events with name /myevents/node/files/filefound. This event should be registered against a node and will trigger against it. The message attached to the event occurrence will contain the values of all parameters passed into orareportevent.
Although this event is fairly straightforward, there are two things wrong::
Any other scripting language or executable program can also be used to implement the logic of an unsolicited event test. However, Tcl is preferable because it allows platform-independent implementation and the fact that the code may be sent from the Enterprise Manager Console `on-demand' without requiring anything to be installed on the Intelligent Agent side.
As with regular Enterprise Manager events, unsolicited events could be triggered only once per condition detected and could clear automatically if the condition that triggers the event is no longer met. Events adhering to this operational pattern are said to have a "proper lifecycle."
Typically, scripts that implement unsolicited events are composed of two basic parts:
The following Tcl script illustrates this two-part script implementation, as well as a technique that allows proper event lifecycle.
#--------------------------------------------------------------------- # # Tcl Procedure # orareportevent1 # # Purpose: # Trigger an unsolicited event only previous state is different # # Arguments: # - event_name: event test to trigger # - severity: new severity # - message: message to be attached to the event report # #--------------------------------------------------------------------- proc orareportevent1 {event_name severity message} { # define a 'lock' that its contents define the previous event status # and figure out the event state during the previous execution global oramsg append event_lock [tempdir] "/" $oramsg(jobid) ".el" if { [file exists $event_lock] } { set f [open $event_lock r] gets $f previous_severity close $f } else { set previous_severity -1 } # if event test state has changed, trigger the event at new severity if { $previous_severity != $severity } { orareportevent $event_name $oramsg(oraobject) $severity $message if { $severity == -1 } { rmfile $event_lock } else { set f [open $event_lock w] puts $f $severity close $f } } } #--------------------------------------------------------------------- # # Event Test Name: # /myevents/node/files/filefound # # Purpose: # Monitor for the existence of a particular file # The test triggers at warning level if the file exists, but # at critical level if the file is larger than the specified # value # # Arguments: # - filename to look for # - critical file size # #--------------------------------------------------------------------- set event_name /myevents/node/files/filefound set file_name [lindex $argv 0] set critical_filesize [lindex $argv 1] if { [file exists $file_name] } { # if the file exists calculate its size in Kilobytes set file_size [expr [file size $file_name] / 1024] if { $file_size > $critical_filesize } { # if file is larger than the critical value, trigger as critical orareportevent1 $event_name 2 "Size: $file_size Kb" } else { # if file is smaller than the critical value, trigger as warning orareportevent1 $event_name 1 "Filesize: $file_size Kb" } } else { # if file in no longer there, clear the event orareportevent1 $event_name -1 "File does not exist" }
This section contains an example of unsolicited event tests. In this case, the test evaluation involves connecting to an Oracle instance and performing some SQL against it.
This example checks the size of a particular table in the database and triggers the event when a set threshold is crossed. There is a warning value and a critical value. The size of the table is measured by counting the number of its rows.
#--------------------------------------------------------------------- # # Tcl Procedure # orareportevent1 # # Purpose: # Trigger an unsolicited event only previous state is different # # Arguments: # - event_name: event test to trigger # - severity: new severity # - message: message to be attached to the event report # #--------------------------------------------------------------------- proc orareportevent1 {event_name severity message} { # define a 'lock' that its contents define the previous event status # and figure out the event state during the previous execution global oramsg append event_lock [tempdir] "/" $oramsg(jobid) ".el" if { [file exists $event_lock] } { set f [open $event_lock r] gets $f previous_severity close $f } else { set previous_severity -1 } # if event test state has changed, trigger the event at the new severity if { $previous_severity != $severity } { orareportevent $event_name $oramsg(oraobject) $severity $message if { $severity == -1 } { rmfile $event_lock } else { set f [open $event_lock w] puts $f $severity close $f } } } #--------------------------------------------------------------------- # # Event Test Name: # /myevents/database/space/tablesize # # Purpose: # Monitor the size of a particular database table # The test triggers at warning level when the warning threshold # is crossed and at critical level when the critical threshold # is crossed # # Arguments: # - table name # - critical threshold # - warning threshold # - username/password for conneting to target (optional) # #--------------------------------------------------------------------- set event_name /myevents/database/space/tablesize set table_name [lindex $argv 0] set critical_threshold [lindex $argv 1] set warning_threshold [lindex $argv 2] if { $argc == 4 } { set connect [format "%s@%s" [lindex $argv 3] $oramsg(oraobject)] } else { set connect [format "%s/%s@%s" $SMP_USER $SMP_PASSWORD $oramsg(oraobject)] } if {[catch {oralogon $connect} lda]} { append msg "Cannot connect to target." "\n" $oramsg(errortxt) orafail $msg } if {[catch {oraopen $lda} cur]} { append msg "Cannot connect to target." "\n" $oramsg(errortxt) oralogoff $lda orafail $msg } set sql [format "select count(*) from %s" $table_name] if {[catch {orasql $cur $sql}]} { append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt) oraclose $cur oralogoff $lda orafail $msg } if {[catch {orafetch $cur} row]} { append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt) oraclose $cur oralogoff $lda orafail $msg } set current_tablesize [lindex $row 0] if { $current_tablesize > $critical_threshold } { orareportevent1 $event_name 2 "Table:$table_name #rows:$current_tablesize" } elseif { $current_tablesize > $warning_threshold } { orareportevent1 $event_name 1 "Table:$table_name #rows:$current_tablesize" } else { orareportevent1 $event_name -1 "Table:$table_name #rows:$current_tablesize" }
A number of OraTcl verbs were used in this script. Refer to the Intelligent Agent User's Guide for details on OraTcl verbs. Note that the preferred credentials, specified in the console, are available to the script writer via the SMP_USER and SMP_PASSWORD Tcl global variables. For jobs against a database, the values of those variables are set to the username and password specified as preferred credentials for that database. Notice that this script allows for an optional overwrite of the preferred credentials via an optional forth input argument.
The Event pane contains the following pages:
You can switch between the pages by clicking the tab of each page. The rows in any page can be sorted on any column by clicking the column heading. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.
The Event pane can be hidden or shown by selecting Show/Hide Event Display in the Console View menu. You can also hide or show the pane by clicking on the flag icon in the Console toolbar.
The Alerts page displays event tests that have been triggered.
Severity of the event occurrence: critical (red flag), warning (yellow flag), clear (green flag), unknown (gray flag), or error state (yellow hexagon).
Name of the event.
Target where the event was triggered.
Time and date of the event occurrence.
Administrators assigned to work on the event occurrence.
Administrator who owns the event.
To view details of an event that has occurred, double-click on the event to display the Event Viewer property sheet. You can enter notes on the nature and progress of the event condition. Note: Comments entered into the log are viewable/editable by admins with the Modify permission. After you have reviewed an event, you can move it to the History page. See "Event Viewer" on page 5-27 for more information.
The Event History page displays a history of events that have occurred and have been moved to History by an administrator or cleared by an Intelligent Agent. The Event History page displays the same columns as the Alerts page.
The History page is refreshed automatically each time you move between the History page and the Alerts or Registered page. However, to refresh the event history list while currently viewing the History pane, you must click the Refresh History icon located at the bottom left of the event History page.
To clear all entries in the History page, click the Clear History icon located next to the Refresh History icon.
The Registered page displays the events that have been registered, or submitted, to monitor test conditions on any network objects. The Registered page contains the following information:
Name of the event.
Target where the event was monitored.
Type of event destination: database, node, listener, web server, Concurrent Manager,
Current registration status of the event: Registered, Registration Pending, De-Registration Pending, and Registration Failed.
Administrator who owns the event.
Note: Under certain circumstances, an event will remain in a Registration Pending state. There are two probable causes:
The Event menu allows you to set up event and administrator information. This menu also provides options to register, track, and view specific events. Menu options are enabled or displayed according to the items selected in the Event pane. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event menu.
When you register or remove an event, there is usually a slight delay while the Intelligent Agent processes the request.
Displays the Event property sheet and allows you to create the definition of a new event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.
Displays the definition of the selected event and allow you to edit the event permissions.
Displays the definition of an existing event. See "Creating, Modifying, and Registering an Event" on page 5-29 for more information.
Acknowledges the selected event in the Alerts page. When an event triggers, an entry is added to the Alerts page. In the severity column, a flag of the appropriate color is displayed along with a pair of eyeglasses. The eyeglasses also appear whenever there is a change in the status of the event (e.g. from `warning' to `critical') If you choose to "acknowledge" this event, then it means you are aware of this event occurrence and hence the eyeglasses will disappear.
Copy the selected event in the Event pane to the Event Library.
Removes the selected event from the Event pane, effectively de-registering the event. This menu option only appears when an event test is selected in the Registered page.
Moves the selected event in the Alerts page to Event History page of the Event pane.
Updates the History pane with the most recent entries.
Clears the contents of the Event History page.
Saves the contents of the Event History page to a report.
Creates a report of currently registered events.
Creates a report of the events that have triggered.
Displays the Event Library dialog. See "Event Library Dialog" on page 5-26 for more information.
If you select an item in the Event pane with the right mouse button, the context-sensitive menu for that item appears. This menu is a subset of the Event menu plus selection-specific menu options.
The Event Library dialog displays the events that have been created and saved to the Event Library. The advantage to using the Event Library is that both events and any associated target information can be stored, copied, or modified in the library for future use. When you create an event, you have the option of submitting, saving to the Event Library, or submitting and saving to the Event Library.
This dialog contains the following information:
Name of the event.
Administrator who created the event.
Select an event and click Edit to display the property sheet for the event. The property sheet allows you to view and modify the events. Editing most event parameters can only be done via the Event Library dialog box.
Several predefined event tests have been installed with Oracle Enterprise Manager. These appear in the Tests page of the Event property sheet. You can add these tests to an event. The tests include:
Only the UpDown tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.
To view the specific tests assigned to an event, double-click on the event in the Event Library dialog and view the Test page of the Event property sheet. See the online help for Oracle events, "Oracle Event Tests" on page 5-40, or the Diagnostic Pack documentation for information on Oracle events and their parameters.
The Event Viewer property sheet displays details on a selected event in the History or Alerts page. When an event triggers, you select the triggered event and bring it up in the Event Viewer. The Event Viewer contains information on why the event triggered. You can also assign the event to the a particular administrator and put instructions for other administrators via the Log page.
You can enter optional comments in the Log page, which is good way to share information about an event with other administrators. Once cleared, events are automatically moved to the History page. The pages of the Event Viewer include:
The Event Viewer General page displays statistics and owner information on a selected event. You can display the property sheet for the event with the Show Event Definition option. The following statistics are displayed on the General page:
Service (database, node, etc.) where the event triggered.
Type of service, such as database, listener, or node.
Time of last update.
Administrator that created the event.
List of administrators that are receiving notifications for this event.
Event test that was evaluated.
Severity of the event occurrence: critical, warning, clear, unknown, or error..
Time and date of the event occurrence.
Message generated from the alert. You may position your cursor above the message to see it displayed as balloon text.
The Event Viewer Log page displays an entry whenever an event is moved to history. An event can be moved manually with the Move to History menu option or automatically when the severity of the event changes.
The Log page also allows comments to be entered on a selected event. Any administrator with permissions to modify the event can add comments in this page. You enter comments in the text box and select the Apply or OK button to add the comment.
The information displayed in the Log page includes:
Text input field allowing you to add comments.
Comment that has been entered for this event.
Administrator that entered the comment.
The date and time when the comment was entered.
The Event Viewer Notification Details displays details of email and paging notifications sent for a selected event. The information displayed in Details page includes:
The severity flag associated with the event occurrence.
Administrator that was notified.
The date and time of the notification.
Method of notification: E-mail or page.
Status of the notification, indicating whether the notification was sent, is pending, or has failed.
If the notification failed, this message indicates the reason for notification failure.
Events include the destination type and the event information that you want to monitor. Events can consist of multiple event tests. To create or modify an event:
If you registered an event, the Intelligent Agent for a destination processes the event and the event appears in the Registered page of the Event pane. Each destination service is listed separately with the event.
There is usually a slight delay between the time the event is registered and the actual notification by the Intelligent Agent.
When threshold values are exceeded for the tests in an event, the event appears in the Alerts page of the Event pane. This notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:
Cases where an event notification is Unknown (gray flag) indicate the target or node where the event is registered is unavailable.
Do not register an UpDown event (included in the Oracle DB Fault event) against the database or node where the Repository schema is stored. If the database containing Repository goes down, the Management Server also shuts down. Hence, the Intelligent Agent cannot inform the Management Server that the database is down.
The property sheet for creating a new event is the same as the property sheet for modifying an event, except that the event name field is read-only. See Figure 5-2, "Event General Page" for an illustration of the Event property sheet.
See "Event Categories and Types" on page 5-10 for more information.
To modify a registered event, perform the following steps:
If the monitored destinations for the event specify multiple targets, e.g. databases A, B, C, then any changes made to the permissions for the Event will apply to all targets, A, B and C.
On the General page, you determine the event name, destination type, description, fixit job, polling frequency, and whether this event should monitor third-party events.
Enter an event name.
Select the destination type you want to monitor from the pull-down list. The types include Database, Listener, Node, or other service that is integrated into the Console.
If the selected Destination Type is "Node", then a second pull-down list of operating systems will appear. If you choose `All', then event tests that apply to all types of nodes, i.e. operating systems, will be available. If you choose a particular operating system, (e.g. Solaris), then additional operating-system specific event tests will be available.
In general, the selection of the Destination Type determines the list of Available Destinations. If your choose "Node" and a particular operating system, such as Solaris, then the list of available destinations will show all Solaris nodes that are running at least an 8.1.7 Intelligent Agent. Any Solaris nodes that use older agents will not be shown.
Enter a description or comment for the event
A fixit job is designed to correct a problem. For example, you may want the Intelligent Agent to run a job to restart a database when the database instance has shut down unexpectedly. Fixit jobs have been created with the Job system and have been designated as fixit jobs. The jobs must be submitted and running on the same destination that the event is set on. You must select destinations to make fixit jobs available in the pull-down list. See "General Page" on page 4-13 for more information.
Fixit job options are:
To turn off a fixit job after an event has been registered, you can copy the registered event to Event Library, remove the event registration, edit the event from the Library to remove the fixit job, and re-submit the event for registration.
Each event must use a unique fixit job on each destination where the event is registered. Also, when a single Intelligent Agent is monitoring multiple databases at a destination, create a separate event and fixit job for each database.
Determine the frequency that you want the test to be evaluated on the selected destinations. The frequency determines how often the event condition is checked. For example, if the frequency is set to 60 seconds, the event condition is checked every 60 seconds. The frequency should not be less than 60 seconds. To set the frequency:
Consider carefully the value for frequency. Some event evaluations may take more time, particularly if there are large numbers of objects to be monitored. The following list covers some of the potentially longer-running database event tests:
* Continued Row and Index Rebuild are resource expensive because they perform an ANALYZE on the segments they monitor.
See "Event Notifications" on page 5-7 for details on the notification frequency.
Check this box to monitor unsolicited events. See "Unsolicited Event Tests" on page 5-12 for more information.
On the Tests page, you determine the event tests that you want to perform. Event test are arranged hierarchically in a tree list for ease of viewing and selection. As with the Console Navigator, you can expand and compress entries in the tree list.
Select the event tests in the list you want to perform in this event, then click on the << (Add) button to move the events to the Selected Events list.
Select the event tests in the list you want to remove from this event, then click on the >> (Remove) button.
The parameter settings for the selected event tests are entered in the Parameters page of the Event property sheet. The settings and types of parameters vary according to the event test selected. Some event tests do not have parameters. See the online help for Oracle events and "Oracle Event Tests" on page 5-40 for information on tests and their parameters.
The parameters for an event are displayed when the event is selected in the Selected Tests list. The parameters vary according to the event selected. Some events do not have parameters.
You can accept the default values or change the values for the parameters. To enter parameter values for an event, you can enter a value directly into a parameter field.
Filtering is used in events such as Chunk Small and Maximum Extents. Examples of filters are = 'SYSTEM', LIKE '%SMP%', and IN ('SYSTEM', 'TOOLS'). Note that the quotes are single quotes. Use uppercase to match the case of the database object names. If you enter a filter value that does not select any objects or is an incorrect value, the event fails.
Determine the permissions that you want to assign to the event with the Permissions Page. This allows other administrators to view or modify the event. Notifications are also assigned with this page.
The levels of permission that you can assign to an Enterprise Manager administrator are:
Does not allow the administrator to view this event anywhere.
Allows the administrator to view the event, inspect event properties, and receive notifications.
Allows the administrator to modify the event's log (See "Event Viewer" on page 5-27), and edit the event's properties except those reserved for Full permission.
Allows the administrator to delete the event, modify permissions for other administrators, and change the ownership of the event.
Allows the administrator to receive enhanced event notifications on the objects through paging or email. Other notifications will be routed to that particular administrator's Console. Notify permission cannot be assigned if the administrator's permission level is set to None.
Any permissions assigned on this page supersede any administrator default permissions. See "Permissions" on page 1-23 for more information. Also, the administrator's notification schedule must be set up in order for them to receive the E-mail/page notification.
When checked, permits external notification (SNMP traps) for third-party event tests at the destination where the Intelligent Agent is located.
Show Notification Schedule displays the notification schedule for the event. The schedule shown on this page is a combined schedule for all administrators that have been given "Notify" privileges for this event. To view administrators assigned to a particular time slot, use the right mouse button to call up the context-sensitive menu, choose the "Remove Recipient" option, and view the list of administrators. To add or remove notifications for an administrator, display the context menu (press the right mouse button) on any time block. The context menu provides options for adding and removing recipients of the notifications.
The Event Progress page displays when you edit an event from the Registered page of the Events pane. This page provides the current registration status for the event selected: Registered, Registration Failed, or Registration Pending. In addition, the destination and time and date when registration was attempted is shown.
When the Progress page is displayed, it shows only the status for the selected event. If the selected event is registered, or had been submitted for registration on other destinations, you can view the status of this event for those destinations by selecting the desired target from the Destination pull-down list. The status of the event displays for that destination. To view the status of this event for all destinations simultaneously, select <All>.
The following options are available on the Progress page:
Select the destination of the event you want to view from the pull-down list. Select <All> for all destinations for which this event has either been registered or failed to be registered.
Status for the event: Registered, Registration Pending, or Registration Failed.
Network destination for the event.
Date and Time the event was submitted for registration.
Displays the Event Status Message dialog. This button is active only when you have selected a failed event registration. Selecting this option will allow you to view the reasons for the failure.
Saves the contents of the list to a text file.
In the following example, a new event for monitoring extents in a database is created and registered at several databases. When creating an event, the General, Tests, Parameters, and Permissions Pages are completed to define the event. You need to register, or submit, an event to run event tests on specific destinations in the network environment.
The following example is used for illustrative purposes. When creating actual events, you must enter accurate values for the polling frequency, and parameter values.
Note: The Maximum Extents event test is only available with the optional Diagnostics Pack. The standard Up/Down event tests provided with Enterprise Manager do not require parameter settings. |
When an event is submitted, each destination is validated and the Intelligent Agent for each destination processes the event. If the registration is successful, the event appears in the Registered page of the Event pane. Each destination database is listed separately with the event.
If an event condition occurs or threshold values are exceeded for an event test, a notification is sent to the Alerts page of the Console Event pane. If administrators have been selected to be notified by email or paging, an email or page is sent. The event notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:
After an event condition is fixed, the event is cleared automatically. There are four event tests that are exceptions to this rule: Alert, Data Block Corruption, Session Terminated, Archiver Hung. Since there is no way to determine automatically that these event conditions have been resolved, these events must be manually cleared.
Oracle Enterprise Manager allows you to specify administrators that are notified when a particular event condition occurs. Each administrator can be associated with an email ID and/or a pager number. When using a paging service or email notification, each administrator can be assigned responsibility for specific systems at specific days and times.
For more information on setting up Oracle Enterprise Manager administrators, see "Managing Enterprise Manager Administrators" on page 1-8.
This section lists the Event system event tests with their parameters and return values. See "Event Parameters Page" on page 5-34 for information on entering parameter values. A list of event tests with numeric pager event Ids is also provided. See "Numeric Pager Job/Event Ids" on page 5-42 for more information.
Event tests are specified for database, listener, and node services. The event tests are also divided into fault, space, resource, and performance management categories. Only the UpDown event tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.
The event test scripts are written in the Tool Command Language (Tcl) enhanced with Oracle Tcl commands (OraTcl). For information on using Tcl and OraTcl to write event scripts, see Oracle Intelligent Agent User's Guide.
Some of the event tests require special tables in the database. For example, the catblock.sql script needs to be run to use the User Blocks event test.
Some of the database event tests, such as Chain Row, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Enterprise Manager Monitor Role" on page 5-5 and "Preferred Credentials" on page 1-25 for more information.
This category of event test tracks severe problems that require immediate attention.
This event test checks whether the database being monitored is running. If this event test is triggered, other database events are not ignored.
none
The Startup Database job task can be set up as a fixit job for automatically correcting the problem.
If the listener serving a database is down, this event test may be triggered because the Intelligent Agent uses the listener to communicate with the database.
This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the Intelligent Agent on a node can be accessed from the Console via periodic "heartbeating" between the Management Server and the Intelligent Agent. Since this event does not actually register with the Intelligent Agent, no fixit jobs can be associated with it. If the defaults are accepted, this event will trigger within five minutes of the Intelligent Agent becoming inaccessible and will clear within three minutes of the Intelligent Agent becoming accessible again.
none
This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the listener on the node being monitored is available.
none
The Startup Listener job task can be set up as a fixit job for automatically correcting the problem.
The Oracle Data Gatherer Alert and UpDown events are both Fault Management events for the node destination type. This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the Intelligent Agent data gathering service on a node can be accessed from the Console. If the Intelligent Agent data gathering service is down, this event is triggered.
none
The Event Management System provides paging services that notify an administrator with a page when an event has occurred. Alphanumeric pagers provide a brief text message identifying the event. Numeric pagers provide the numeric pager event Ids to identify the event.
For job notifications, you will receive a 6 digit number. The first 3 digits indicate the job-id. The last 3 digits indicate job status.
For event notifications you will receive the event ID with the status code.
For a complete list of pager job/event IDs, see "Paging Status Codes for Numeric Pages" on page 1-18
The following procedures will assist you and/or Oracle Support if problems are encountered with the Event subsystem.
If you need to contact Oracle Support, you must provide specific information on the status of your Oracle Enterprise Manager repository. To assist you, a SQL script has been provided that extracts pertinent Repository data and generates a debug log file.
To generate the log file, perform the following procedure:
$ORACLE_HOME/sysman/log.
connect <repository name>/<repository password>
@$ORACLE_HOME/sysman/admin/evtdbg.sql
Because the Enterprise Manager framework is a three-tier system that can manage a heterogeneous environment, it is important to keep in mind various software version requirements necessary for proper event system operation. The following table lists event system features and associated software version requirements.
Feature Name | Description | Enterprise Manager Version | Required Agent | Management Server/Console Required | Works In Browser |
---|---|---|---|---|---|
Advanced Events |
All events for databases, nodes, listeners. See Enterprise Manager online help for more information. |
Diagnostics Pack 1.5.5 and highe.r |
All supported agents, latest recommended |
For EM 2.x, the Management Server and Console that corresponds with that Pack |
yes |
Event Handler |
Component that allows you to log event information or execute custom commands in response to an event occurrence. See Appendix A, "Enterprise Manager Event Handler" for more information |
2.0.4 and higher/ beta |
n/a |
2.0.4 OMS |
n/a |
|
|
2.1 and higher / production |
n/a |
2.1 OMS |
n/a |
Improved Node Up/Down Monitoring |
Enhancement to the Node Up/Down event test. Provides more information on whether or not the node is down, the agent is down, etc. |
2.2 and higher |
all supported |
2.2 and higher |
yes |
User-Defined SQL Test |
Allows you to write your own custom SQL to monitor database events |
Diagnostics Pack 2.1 and higher |
8.1.6 and higher |
2.1 and higher |
yes |
Enhanced monitoring for target subcomponents |
"For events whose targets involve multiple subcomponents (e.g. monitor tablespace full for ALL tablespaces), information on which subcomponent is in alarm is now provided |
"2.2 and higher |
8.1.7 and higher |
2.2 and higher |
yes |
Context sensitive help for Event tests |
"In the Parameters tab of the Event dialog, invoking ""Help"" will bring up information pertinent to the current selected event test |
"2.2 and higher |
n/a |
2.2 and higher |
yes |
Events with synonymous event tests |
"Events can be created that have more than one of the same event test (e.g. a ""Tablespace Full"" test for ""SYSTEM"", another ""Tablespace Full"" event test for ""USER"") |
"2.2 and higher |
all supported by EM 2.2 |
2.2. and higher |
yes |
Job and Events Notification filters |
|
|
|
|
|
Allows you to filter pages & emails based on job and event status |
2.1 |
all agents supported by EM 2.1 |
2.1 |
yes |
|
Allows separate filters for pages & emails based on job and event status |
2.2 and higher |
all agents supported by EM 2.2 |
2.2 and higher |
yes |
|
Customization for paging & email messages |
Allows you to customize the messages for email and pages |
Diagnostics Pack 2.2. and higher |
all agents supported by EM 2.2 |
2.2 and higher |
yes |
Advanced O/S event tests |
New event tests that monitor operating system specific metrics |
2.2 and higher |
8.1.7 and higher |
2.2 and higher |
yes |
|
Events to monitor error conditions against the Oracle Applications Concurrent Processing Server
|
|
|
|
yes |
|
Events to monitor error conditions against the Oracle Developer Forms Server
|
2.0.4 and higher (2.0.4 console needs Forms Extensions.
|
8.0.6 and higher (requires Forms Agent Extensions.
|
2.0.4 and higher (2.0.4 console needs Forms Extensions.
|
yes |
Program Filtering in Concurrent Manager Events
|
Allows you to monitor particular Oracle Applications Concurrent Programs. Also allows you to exclude particular Concurrent Programs from being monitored.
|
|
|
|
yes |
|
![]() Copyright © 1996-2000, Oracle Corporation. All Rights Reserved. |
|