Release 3 (8.1.7)
Part Number A83729-01
 Library |
 Product |
 Contents |
 Index |
| Oracle8i Integration Server Overview Release 3 (8.1.7) Part Number A83729-01 |
|
This chapter contains:
This section examines several Oracle products that are involved in Oracle Integration Server. It contains:
Advanced Queuing (AQ) is a database-integrated message queuing component of the Oracle8i Enterprise Edition database management system. It provides an infrastructure that simplifies the task of passing messages within an application or between applications.
Its functional strong points include:
Advanced Queuing has several key components:
A message is the smallest unit of information inserted into and retrieved from a queue. A message consists of:
The control information represents message properties used by AQ to manage messages. The payload data is the information stored in the queue and is transparent to Oracle AQ. A message can reside in only one queue. A message is created by the enqueue call and consumed by the dequeue call.
A queue is a repository for messages. There are two types of queues: user queues, also known as normal queues, and exception queues. The user queue is for normal message processing. Messages are transferred to an exception queue if they cannot be retrieved and processed. Queues can be created, altered, started, stopped, and dropped by using the Oracle AQ administrative interfaces.
Queues are stored in queue tables. Each queue table is a database table and contains one or more queues. Each queuetable contains a default exception queue.
An agent is a queue user. This could be an end user or an application.There are two types of agents:
Any number of producers and consumers may be accessing the queue at a given time. Agents insert messages into a queue and retrieve messages from the queue by using the Oracle AQ operational interfaces.
An agent is identified by its name, address and protocol.
name of the agent may be the name of the application or a name assigned by the application. A queue can itself be an agent, enqueuing or dequeuing from another queue.
address field is a character field of up to 1024 bytes that is interpreted in the context of the protocol. For instance, the default value for the protocol is 0, signifying a database link address. In this case, the address for this protocol is of the form
queue_name@dblink
where queue_name is of the form [schema.]queue and dblink can be either a fully qualified database link name or the database link name without the domain name.
The recipient of a message may be specified by its name only, in which case the recipient must dequeue the message from the queue in which the message was enqueued. The recipient may be specified by name and an address with a protocol value of 0. The address should be the name of another queue in the same database or another Oracle8i database (identified by the database link) in which case the message is propagated to the specified queue and can be dequeued by a consumer with the specified name. If the recipient's name is NULL, the message is propagated to the specified queue in the address and can be dequeued by the subscribers of the queue specified in the address. If the protocol field is nonzero, the name and address field are not interpreted by the system and the message can be dequeued by a special consumer.
A single message can be designed for consumption by multiple consumers. There are two ways to do this:
name, address and protocol.
Different queues can have different subscribers, and the same recipient can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby overriding the subscriber list.
A rule is used to define the interest of one or more subscribers in subscribing to messages that conform to that rule. The messages that meet this criterion are then delivered to the interested subscribers. Put another way: a rule filters for messages in a queue on a topic in which a subscriber is interested.
A rule is specified as a Boolean expression (one that evaluates to true or false) using syntax similar to the WHERE clause of a SQL query. This Boolean expression can include conditions on the following:
A rule-based subscriber has rule associated with it in the default recipient list. A rule-based subscriber is sent a message that has no explicit recipients specified if the associated rule evaluates to TRUE for the message.
The queue monitor (QMNn) is a background process that monitors the messages in the queues. It provides the mechanism for message delay, expiration, and retry delay. The QMNn also performs garbage collection for the queue table and its indexes and index-organized tables.
It is possible to start a maximum of 10 multiple queue monitors at the same time. You start the desired number of queue monitors by setting the dynamic init.ora parameter to aq_tm_processes. The queue monitor wakes up every minute, or whenever there is work to be done, for instance, if a message is to be marked as expired or as ready to be processed.
AQ uses the strength of the Oracle8i database management system (DBMS) to provide these general features:
Messages are placed in normal rows in a database table, and so can be queried using standard SQL. This means that you can use SQL to access the message properties, the message history, and the payload. All available SQL technology, such as indexes, can be used to optimize the access to messages.
Standard database features such as recovery, restart, and enterprise manager are supported. Oracle AQ queues are implemented in database tables, hence all the operational benefits of high availability, scalability, and reliability are applicable to queue data. In addition, database development and management tools can be used with queues. For instance, queue tables can be imported and exported.
Users can use object types to structure and manage message payloads. RDBMSs in general have had a far richer typing system than messaging systems. Since Oracle8i is an object-relational DBMS, it supports both traditional relational types as well as user-defined types. Many powerful features are enabled as a result of having strongly typed content, such as content whose format is defined by an external type system. These include:
Users of AQ can specify that messages be retained after consumption. The systems administrator can specify the duration for which messages will be retained. Oracle AQ stores information about the history of each message, preserving the queue and message properties of delay, expiration, and retention for messages destined for local or remote recipients. The information contains the ENQUEUE/DEQUEUE time and the identification of the transaction that executed each request. This enables users to keep a history of relevant messages. The history can be used for tracking, data warehouse and data mining operations.
If messages are retained, they can be related to each other. For example, if a message m2 is produced as a result of the consumption of message m1, m1 is related to m2. This allows users to track sequences of related messages. These sequences represent event journals, which are often constructed by applications. Oracle AQ is designed to let applications create event journals automatically.
The integration of control information with content (data payload) simplifies application development and management.
With Oracle8i, an owner of an 8.1 style queue can grant or revoke queue- level privileges on the queue. DBAs can grant or revoke new AQ system level privileges to any database user. DBAs can also make any database user an AQ administrator.
AQ can deliver non-persistent messages asynchronously to subscribers. These messages can be event-driven and do not persist beyond the failure of the system (or instance). AQ supports persistent and non-persistent messages with a common API.
A combination of features enable a publish-subscribe style of messaging between applications. These features include rule-based subscribers, message propagation, the listen feature, and notification capabilities.
Oracle AQ offers two development contexts:
Java using the oracle.jms Java Package. This implementation of a public standard extends the defined W3C interfaces so that developers operating in the JMS context have the same means as those working within native AQ.
A comprehensive graphical interface supports the administration of Advanced Queuing objects through the Enterprise Manager DBA Studio (Schema Management), part of the DBA Management pack, which is in the 8.1.7. database release.
|
See Also:
|
Oracle Corporation has extended the JMS API to leverage those Oracle Advanced Queuing features that cannot be accessed through a standard JMS API. We are working with the standards body to include these extensions in the JMS standard.
We have extended the standard JMS API in the following areas:
The standard JMS definition expects each message to be delivered to a single recipient. Because AQ can record, manage, and track the delivery of a message to more than one recipient without having to copy the message, the OJMS API supports the use of agents.
An agent is associated with the message for each recipient of the message. The agent is an object that has a name, a transport protocol, and an address that is specific to the transport protocol.
The agent model is particularly important for the publish-subscribe capability of AQ and for the interoperability of AQ with other messaging technologies.
To use standard AQ features that are not supported by the standard JMS message properties definition, OJMS has extended the message control property set to include:
AQ is unique among messaging technologies in supporting strongly typed message payloads that you can access using standard SQL while they are in the queues and that support content-based routing. AQ uses Oracle Object Types (also known as ADTs) to do this. To support this feature, OJMS has an additional message type called an ADTMessage.
AQ messages can be created within the context of an Oracle database COMMIT transaction. The same transaction can include SQL operations such as table inserts and SAVEPOINTs without requiring a two-phase or XA-compliant commit.
We extended the JMS Transacted Session feature to support this unique Oracle feature, which is useful in developing adapters to interface to Oracle-based applications.
The JMS standard includes a concept of administered objects. Chapter 4, "Key Integration Concepts", describes Oracle JMS implementation in depth. The Oracle JMS implementation has an API for creating and managing these objects.
The Session interface now supports administrative functions such as the creation of Queue Tables and Queues and Topics and the modification of their properties.
The Destination interface now supports administration of Queues and Topics and AQ propagation of messages between destinations. This has significance to a hub and spoke architecture that is used for the hardware configuration, security model, or persistence schema.
The first release of the OJMS interface (with release 8.1.6 of the database) has a small number of standard JMS features that are not supported or are supported in a non-standard manner:
Oracle Procedural Gateway for IBM MQSeries enables Oracle applications to create MQSeries messages directly or to convert AQ messages into MQSeries messages and vice-versa. It has a graphical interface called Visual Workbench that you can use to define the mapping of Oracle object types to MQSeries structures and to import Oracle Object Types and COBOL Copybook definitions.
You then complete the configuration by generating PL/SQL packages that provide a seamless transactional API to the MQSeries queue that you can access from SQL*Plus, PL/SQL packages, Pro*Series languages, and so on.
TIBCo sells an adapter that you can use to transform AQ messages into TIB Rendezvous messages and vice-versa. The product was developed in collaboration with Oracle and provides a code-free transformation solution between Oracle Advanced Queuing and TIB Rendezvous (TIBrv).
The software is designed to convert messages from AQ to TIBrv, to transport them across the TIB bus, and to convert them back into AQ without losing any of the AQ message properties. Messages are converted transactionally using a version of the TIB Rendezvous Certified Messaging and the Oracle commit model.
This section examines the ways in which these products interface in an integration solution. It contains:
Within an integration solution based on business events, the AQ message queuing functionality provides the backbone for the solution and supports business event requirements not previously expected of messaging technologies such as:
Advanced Queuing supports business events by using the Oracle8i database to provide a repository for messages, transactional support, and recovery. Most of the other products in the Oracle Integration Server can be accessed directly through a delivered AQ interface. This reinforces the position of AQ as the backbone of the solution.
AQ keeps a history by determining and recording the recipients of the message when the message is created. Each time a recipient attempts to collect the message, the attempt is recorded automatically. This guarantees that a message is collected only once by each recipient and that you always know when a message has been collected.
AQ messages have three main components: a header, a payload, and a history. All three are stored in queue tables that consist of a set of regular database objects in the database. Provided that the database is protected by a suitable backup and recovery routine, messages can be recovered to a consistent state in the event of a system failure.
AQ messages are accessed through an API that prevents the message header, payload, or history from being changed directly by the user accessing the message. This design feature, when combined with the retention of messages after they have been consumed, enables you to construct an audit trail.
Although AQ uses a sophisticated set of database objects to implement this rich functionality, it also includes a set of views for each queue table so that you can easily interrogate the header, payload, and history of the messages using SQL.
If the payload structure is defined as a simple object type, SQL can access the attributes directly. However, if a generic object type containing a CLOB is used to store the payload in, for example, XML format, then the character string must be translated into a more strongly typed structure before it can be interrogated easily.
As already mentioned, AQ uses a set of regular database objects to implement the logical construct of a queue table. We have engineered this set of objects to ensure that retaining consumed messages does not adversely affect the creation or collection of other messages.
The ability to determine the recipients of messages according to predefined rules is comprehensively supported by Advanced Queuing.
Multiconsumer queues provide publish-subscribe routing for durable subscribers:
tab.user_data.
You can also use propagation to route messages to groups of recipients within a publish-subscribe context as shown in the following scenario:
Red Cars.
Red Cars subscribes to Queue Sports Cars with a rule of tab.user_data.color = RED.
Sports Cars.
RED, Queue Red Cars is recorded as a recipient of the message.
Red Cars and records all the recipients on the subscription list as recipients of the propagated message.
Advanced Queuing is the most appropriate mechanism for routing if the subscribers are durable but change frequently (daily or more often).
If the subscription is static and the rules for routing are complex, Workflow (Chapter 10, "Workflow") provides a better alternative to develop your solution.
If Oracle Replication does not provide the required functionality, use Advanced Queuing technology to develop a predefined replication environment.
You can do this in two ways:
Developing your own replication software using AQ requires a special understanding of replication issues and a commitment to maintain and upgrade predefined software.
If you use Java as the development language for the programmable aspects of the solution and if you employ JMS as a standard for communicating with messaging technologies, then OJMS offers an effective API to Advanced Queuing.
In a B2B-oriented solution, you can use AQ features additional to those included in the JMS standard, such as object type payloads, transacted sessions, and the agent model. The OJMS extensions to the JMS standard make it possible to use these extra AQ features. If you use these features, OJMS offers an excellent alternative to the JMS implementation of Oracle Message Broker.
The Procedural Gateway for MQSeries is the only gateway to another messaging technology offered by Oracle.
Use it to transform messages between MQSeries and Advanced Queuing. In its current form, this requires you to develop a simple PL/SQL bridging procedure to call the MQSeries dequeue package or the AQ enqueue API. It supports only the conversion of simple payloads and does not support collections or LOBs.
The message throughput across a single queue bridge varies depending on system specification and message size. Do not choose Procedural Gateway for MQSeries if you anticipate hundreds of messages per second.
The products that make up the Oracle Integration Server interface smoothly with Advanced Queuing to provide asynchronous services comparable to those provided by many third-party application integration tools.
However, some applications require interaction with other messaging technologies, such as MQSeries and Rendezvous from TIBCo, because of a decoupling requirement, an application platform, an existing interface, or a previous choice of strategic technology.
If the application has an interface that extracts messages to a messaging technology other than AQ, you must decide whether to integrate the messaging technology to AQ, OMB, or Workflow directly by using a programming language or through specialized software.
Alternative methods for integrating to Advanced Queuing are:
Programmatic 1: If both message queuing products have an API for a particular language, write a program that collects the message from one product, transforms the message properties, and creates the message in another product.
Programmatic 2: Choose two languages that can communicate with each other and that have APIs to message queuing products.
Software: Choose a specialized product that enables conversion between the two messaging products.
The example given here is used solely to illustrate the process and should not be taken as a recommendation of any one solution over any other.
Scenario
One of the applications included in a hub-and-spoke integration solution has a ready-made message-based interface that places messages in a set of MQSeries queues and that expects business events in the form of MQSeries messages. The application interface is written in C and creates and consumes messages using the native MQSeries API.
The objective is to transport messages from MQSeries to AQ and from AQ to MQSeries.
The solution depends upon the architectural principles, the language or languages used in the integration solution, the approach to modularization, the standards used, and so on.
If the second message queuing product is TIB Rendezvous from TIBCo rather than MQSeries, then the scenarios are similar except that TIBCo and not Oracle is the software provider and the software is called The TIB Adapter for Oracle.
As discussed earlier in this chapter, Advanced Queuing can manage two types of queues: persistent queues and nonpersistent or volatile queues.
This section contains:
Persistent queues are ideally suited to manage messages that flow between applications (when integrating applications with an enterprise) or between two business processes (when integrating businesses for B2B commerce). There are two reasons why such messages must be stored persistently:
Particularly for B2B commerce, auditing and tracking of messages is a critical requirement.
The message payload of AQ can also be an unstructured data type such as RAW or BLOB for flat-file-oriented data such as SAP TDOCs and XML-based business object documents. Consider three issues when warehousing message payloads:
RAW or BLOB format. If you receive B2B messages as XML payloads, you must decide how completely to parse the payload. If you must perform analytical operations frequently, you must parse the payload. If the payload is large, construct a DOM parse tree; if the payload is small, use a SAX API. (Typically a payload that is greater than 100 MB is considered large.) If only a small set of fields within the XML document require analysis, then you extract only those fields from the parse tree and store them in tables in SQL format. Retain the rest of the payload in RAW or BLOB format.
For more information on the Oracle XML facility, read the Oracle8i XML Reference Guide Release 3 (8.1.7)
SORT and SUM operations on the data in the partition for that day.
The messaging infrastructure must maintain a persistent message store to guarantee message delivery even when message propagation or transport fails.
In contrast to a persistent messaging environment, volatile queues are ideal for best-effort delivery of messages. For instance, a message sent to confirm that a B2B message has been received does not need to be stored persistently.
As previously described, each message passed through AQ is stored in a queue. You can store the contents of single or multiple queues in a queue table. Four basic principles govern the storage of messages for business analysis and intelligence:
The Oracle Business Intelligence Tools are a set of three Oracle products that provide simple, intuitive access to business information in Oracle databases. Use them to analyze business intelligence stored in message warehouses.
Oracle Reports, Oracle Discoverer, and Oracle Express have graphical interfaces that can be accessed from anywhere through a standard Internet browser. They deliver decision support information to business users (usually from a data warehouse or data mart). Recently, we've enhanced them to enable operational monitoring for throughput analysis and service level monitoring in an asynchronous messaging integration solution.
The tools can be used to generate and view dynamically created reports, to perform dynamic queries against virtual and materialized data structures, and to analyze and forecast trends.
Oracle Reports is a graphical tool for developers of sophisticated, high-quality reports. Its rich functionality enables reports with complex hierarchic structures and matrices to be defined quickly and easily, using the full power of SQL.
Users can then view these dynamically generated reports through a standard web browser.
Oracle Discoverer is an easy-to-use browser-based tool, primarily aimed at business users, that can be used to perform dynamic queries, to provide reports and charts, and to enable Web publishing.
It employs the End User Layer, an abstraction layer over database objects, to simplify queries and enable predefinition of queries. It provides drag-and-drop functionality that removes the need for the user to be versed in the grammatical constructs of SQL.
Powerful, integrated charting illustrates trends and exceptions and enables users to drill down through charts to view specific data more closely.
Some of its more sophisticated features include:
Oracle Express is a decision-support tool with an online analytical processing (OLAP) engine. It employs a multidimensional data model that expands rows and columns into multiple categories of data called dimensions.
This type of data model is optimized for decision support, uses array arithmetic to provide quick access to the data for "slice and dice" type queries, and offers built-in functions for analysis, forecasting, modeling, and what-if scenario playing.
The data model can be interrogated using intuitive tools that are accessible through a standard Web browser. The OLAP engine can use a data cache to store dimensional data as an aid to query performance.
|
 Copyright © 1996-2000, Oracle Corporation. All Rights Reserved. |
|