- XSLT transformations in OSB are used for XML-to-XML mappings. You can test them with source XML formats within the test console. The test console displays named parameters supported by the transformation.
- XQuery transformations work with either XML or non-XML input
- MFL is used to describe non-XML data formats and the layout of binary and text data in these formats. The test console can verify the correct bidirectional mapping.
My vision is the closed loop between business and IT (both on the same page). The Moebius loop perfectly symbolizes how different sides can "melt". Overcoming cultural, language and collaboration barriers while changing the role of IT is a prerequisite for successful digital transformation in my eyes.
Monday, June 11, 2012
Interactive OSB Testing
Thursday, March 29, 2012
JMS MaxMessage Size - persistent messages
Large JMS Messages in WebLogic
| Applies to | WLS 10.3.5 / OSB 11g - plain vanilla setup, 64bit, 3GB Heap |
| Requirement | Send large messages (250MB) across OSB using JMS Transport |
| Background | OSB supports streaming mode for HTTP and File-based protocols - which we tested successfully with multi-GB files. However, things are not so straightforward with JMS proxies which essentially boils down to the WLS JMS subsystem. |
| Test Setup | The test is based on a JMS client that creates a persistent message with a variable payload size (1-250 MB). It sends this message to a JMS queue on a remote WLS server and receives it synchronously from the same queue. We measure the roundtrip time on the client as well as server load (heap, CPU) on the WLS server. |
First Run
At first it looks good - you'd get a near linear increase in response time in relation to the message size. But then - at 10MB size - you will encounter the first roadblock:
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: ’10000080′ bytes exceeds the configured maximum of: ’10000000′ bytes for protocol : ‘t3′.
Increase MaxMessageSize
For most protocols, including T3, WLS limits the size of a network call to 10MB by default (see Performance and Tuning Guide). The 10MB limit is a soft limit meant as a protection against DoS-attacks. It is worth noting, that a server is more prone to DoS attacks when increasing this limit. Anyway, let's increase it to see how large a message we can process.
- Go to Managed server--> Protocols --> General and set the Maximum Message Size to something large.
- To set the maximum message size on a client, use the following command line property: -Dweblogic.MaxMessageSize
- Verify the Queues maximum accepted message size:
Queue-> Configuration-> Threshholds and Quotas-> Maximum Message Size
Setting Maximum Message Size for Network Protocols
Increase Server Heap Size
When running the test again you'll notice that the server-side heap is really busy on those large messages.
Out heap size of max 3GB seems to be a limiting factor. So we increase it to 8GB (note: you need a 64-bit JVM for anything larger than 3GB):
USER_MEM_ARGS="-Xms3096m -Xmx8192m -XX:PermSize=128m -XX:MaxPermSize=512m -XX:NewRatio=3 -XX:SurvivorRate=128"
And off we go with our next test. This time we can send up to around 100MB messages. Roundrip times increase up to 50-80 seconds. Then we get a CORBA exception:
weblogic.jms.common.JMSException: weblogic.messaging.dispatcher.DispatcherException: java.rmi.MarshalException: CORBA COMM_FAILURE 1398079691 Maybe; nested exception is:
org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 203 completed: Maybe ...
Bypass CORBA Stack
At this point we got a hint from Oracle support: "Switch from wlclient.jar to wlthint3client.jar so you bypass all the CORBA stack…". This change guarantees that the T3 protocol is used for JMS communications.
We can now send messages up to around 220 MB of size - albeit at extremely long roundtrip times of around 10 minutes. Try anything larger and you get:
weblogic.jms.common.JMSException: weblogic.messaging.dispatcher.DispatcherException: weblogic.rjvm.PeerGoneException: No message was received for: '240' seconds
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:116)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncNoTran(DispatcherAdapter.java:61)
at weblogic.jms.client.JMSSession.receiveMessage(JMSSession.java:894)
at weblogic.jms.client.JMSConsumer.receiveInternal(JMSConsumer.java:647)
at weblogic.jms.client.JMSConsumer.receive(JMSConsumer.java:511)
In-flight Compression
WLS JMS has another nice feature: online compression of messages during transport (see
Compressing Messages). Since our test messages contain EDIFACT sample payloads they can be reduced to around 10% of their original size. This also has a dramatic effect on roundtrip times, as the chart below shows (up to 100MB). 250 MB messages can now be easily transferred and take less than 10 seconds for roundtrips.
Conclusion
We didn't crack the 200MB limit on physical message sizes in WebLogic JMS. However, we found a solution to our particular requirement. It would be interesting to find out, how to get rid of the PeerGoneException in Test-03. Even though, the roundtrip times for this kind of message size are prohibitively long.
Wednesday, November 30, 2011
Oracle ADF Powerkurs - 15. Dezember in Bern
Am 15. Dezember findet in Bern ein 1-tägiger Powerkurs zum Thema Oracle ADF Entwicklung statt. Durchgeführt von Frank Nimpius, dem Experten der Oracle für das Thema. Weitere Details und Anmeldung unter: http://www.ipt.ch/veranstaltungen/javaee-applikationsentwicklung/.
JavaEE-Applikationsentwicklung
Schnell und effizient mit Oracle ADF
Am 15. Dezember 2011 in Bern
Monday, October 31, 2011
First-class citizens and splendid isolation
Thomas Rischbeck says: “The ESB pattern is about performing integration tasks and adding value to client-service communication in an SOA – all completely transparent to the participants”. I would welcome this transparency if it were not based on intelligent routing and data enrichment having its own policies on the interaction between the participant. In SOA, we have a mechanism that regulates the interactions – Service Contract – and it is not seemed to be considered by the ESB at all.
Michaels seems to imply that the ESB doesn't honor service contracts at all. I get the impression that Michael also sees an ESB as a magical integration switchbox in the middle of the enterprise. This makes Michael feel uneasy – which I can understand very well.
There’s also another way of looking at an ESB: You could think about it as an opaque substrate for hosting "facade" services – which are fully-fledged, first-class citizens in the SOA world. Like every other service, such facade services have a formal interface (and a less formal contract which is a superset of the interface (as James Watson says. While I’d admit that there can be quibbles about business logic on the ESB – it doesn’t “break […] the service contract between the service consumer and the service provider”, as Michael thinks. The consumer wouldn’t’ reason about the actual backend provider but only about the ESB-exposed endpoint it communicates with. The rest is implementation details, so to say.
From this point of view the ESB becomes opaque. It cannot violate a service contract because it isolates whatever backend services are multiplexed through the ESB. This splendid isolation renders numerous benefits. For example, services can be normalized according to enterprise requirements; they may also enforce coarse-grain security and shield backend services from certain attack vectors. Many rings of defense is a security best practice since the middle ages.
Cheers, Thomas.
Saturday, November 06, 2010
Ride the Cloud at Lightning Speed
- Partition the workload into smaller items
- Map these onto processors or machines
- Communicate between these subtasks
- Synchronize between subtasks as required
Even with the use of toolkits that offer communication and synchronization primitives (such as PMI) parallel programming is a bit like rocket science. Without a shift in programming model parallel programming will stay elusive. Programmers will have to deal with parallelism to some degree at least – they can no longer rely on middleware, databases and operating systems to exploit parallelism.
- Actor Model: Dispatching and scheduling instead of direct invocation Queues and asynchronous interactions
- RESTful interactions Message passing instead of function calls or shared memory
- Eventual consistency instead of ACID
Friday, October 08, 2010
Welcome to Giovanni Scheepers - Asset Liability Management for SOA
Sunday, August 22, 2010
Offroad Google
Saturday, June 12, 2010
SOA Symposium 2010 in Berlin - CFP
Modern Service Technologies & Practices
• Service Governance & Scalability
• SOA Case Studies & Strategic Planning
• REST Service Design & RESTful SOA
• Service Security & Policies
• Semantic Services & Patterns
• Service Modeling & BPM
Your Business & Infrastructure Into the Cloud
• The Latest Cloud Technology Innovation
• Building and Working with Cloud-Based Services
• Cloud Computing Business Strategies
• Case Studies & Business Models
• Understanding SOA & Cloud Computing
• Cloud-based Infrastructure & Products
Friday, June 11, 2010
SOA Forum 2010 - Asset-based SOA
As keynote speaker Massimo Pezzini of Garnter pointed out: assets come in many flavours. They can be infrastructure (SOA backplane), documentation, process templates, organizational templates, skills ... and people! Massimo differentiated three different SOA-flavours: RPC, WOA (request-driven, REST) and Event-driven SOA.
Keynote speaker Anne Thomas Manes talked about the reincarnation of SOA. 18 months ago Anne published a eulogy on SOA. This eulogy wasn't praise more than lamentation at the demise of a promising development - that met it's untimely dead because of hype and wrong application. A talent wasted?
Well - SOA's resurrection happened. Although the connotation here is somewhat religious. Anne suggests to avoid all religious and fundamentalist mindsets when applying technology and architecture. SOA is a logical choice and sine-qua-non for many problems. But it's not a panacea. An opinion that was reflected in the panel discussion.
The wrong approach to SOA - according to Anne - consisted of too strong a technology focus (ESBs, appliances, WS-stacks, governance tools and so on) and too little focus on architecture. Frequently systems were built with an integration mindset which is akin to putting lipstick on a pig (Anne has a lovely picture in her slidedeck). Ultimately this led to high maintenance costs. According to her figures the development of a system is only 7-15% of the TCO - the rest is maintenance. IT budget cuts mean that innovation comes to a screeching halt because there is not much opportunity to reduce maintenance costs.
The reincarnation of SOA consists of seeing the big picture rather than technologies and point solutions. She recommends to learn from the Web 2.0 technologies and help enterprise software break out from the firewall. Externalization, consumerization and democratization of IT are the keywords here. A sentiment echoed by Zapthink and others.
SOA's reincarnation is driven by the application of design principles and architecture instead of focussing on technology. SOA principles are applied through:
- Modular
- Distributable
- Discoverable
- Swappable
- Shareable
The SOA Forum presentations can be accessed under the following URI:
Friday, June 19, 2009
ESB Book - New Cover
The book "Modern ESB Architecture for SOA" I am writing together with Mark and Arnaud now has a new and final cover.
Friday, May 15, 2009
SOA Forum 2009 is history
Saturday, May 09, 2009
End-of-time Prophecies on Middleware, SOA, ESB and else

When Nostradamus pronounced and published his famous prophecies in the 16th century he did not need to worry about target consumer categories. His predictions - some of them interpreted as end-of-time prophecies or just plagues, earthquakes, etc. - where all-encompassing. Unfortunately, none of his writing was ever interpreted as predicting an event before it actually ocurred. :-)
So, what about those target consumer categories? I came across these when reading an excellent article by Steve Vinoski with the title "Is Middleware Dead? (IEEE Internet Computing, vol. 8, no. 4, 2004, pp. 94-96). This article is also about predictions -- in the technology and market domain this time.
Steve Vinoski wrote this article in response to "The End of Middleware", where Jonathan Schwartz of Sun touted middleware to be dead. The assumption behind this pronounciation: everybody would use Sun's Java Enterprise System then there would be no need to integrate with other middlewares. Of course, it did not pan out this way.
Similar pronouncements about the death of CORBA, SOA, ESB and many other technologies have been made. Steve points out that such predictions frequently do not take the target consumer category into account. According to Crossing the Chasm, there are five such categories in a technology adoption lifecycle (- innovators, - early adopters, - early majority, - late majority, - laggards).
When an "early adopter" feels that a certain technology is dead -- it may be alive and kicking from the point of view of the "laggard". One-size-fits-all solutions are therefore impossible. Where are ESBs on that adoption bell curve? Pretty much at (or slightly after) the peak, I'd say.
Wednesday, May 06, 2009
Stateful vs. Stateless Services
Clearly, stateless services do scale better: Additional instances of the service can be fired up to cope with increased load. They can also deal better with failover situations. A client request may be routed to an alternative instance transparently when the original instance goes down.
However, sometimes stateful services do make sense from a modelling perspective. Nicolai Josuttis, for example, describes a shopping cart service. For this type of service, a client would expect to successively call operations to add new items.
A stateful service may be implemented as a stateless service: either by sending all relevant information (eg the shopping cart contents) with every request or by sending a session ID with every request and keeping the shopping cart contents in the DB. Or a "real" stateful service keeps the shopping cart state in memory - this requires session affinity: ie, each successive request by a given client must be routed to the same service instance. When this instance goes down, the user must start their shopping trip all over again -- possibly an acceptable tradeoff in an eCommerce application. Nicolai Josuttis works out the associated tradeoffs (state in frontend, state in service, state in backend) really nicely in his book (p.195).
Tuesday, April 28, 2009
Oracle aims for a Slice of Cake
Monday, April 27, 2009
Message-Driven SOA = Robustness
In the past I've worked as architect on an eGovernment project that followed just these same architectural principles. That was by design and also by necessity -- since the underlying product (SonicESB) naturally guided developers into modelling services as entities with a purely message-oriented interface (of course, that was putting the horse before the cart). Services in SonicESB aren't really services in the conventional sense -- they are written against an API and can communicate with other Sonic services only using that API. The API binds onto a JMS abstraction layer that exercises the underlying MOM implementation. However, some queues that drive services may be exposed via Web service interfaces to the external world. Sonic also offers a light-weight process construct: itinerary-based routing includes a routing slip with every message and describes which queues/services must be traversed in what order.
The project team really bought into the benefits of temporal decoupling and came to see it as godsend -- they also came to accept the unusual asynchronous communication style that goes with it.
- individual service may be taken down for maintenance without negatively affecting the rest of the system -- messages simply get buffered until the service comes up again
- as a corollary to the above: services do not need to handle peak loads -- just the average load. When bursts of messages arrive they are simply backlogged in the queue until the service can process them.
- non-blocking for the message producer: fire-and-forget with reliable messages; producers "know" that messages eventually arrive - SLA of the recipient is can help in giving a deterministic timeframe
- monitoring of queues: analyze queue utilization over time; detect usage patterns and adapt to and/or predict load situations;
- inspection of in-flight messages: analysis of messages; remove and store individual messages; replay them at a later stage (or for testing); isolate poison messages and use them for failure analysis
Thursday, April 23, 2009
Virtualization at the Endpoint?
Service virtualization is nothing new, of course - it's a pattern that underlies most SOA middleware products (ESBs) and agents (Actional, AmberPoint,..). Hosting platforms (WCF, App Servers) also provide rudimentary virtualization on top of the business logic.
The strength of virtualization lies in its power to isolate consumer and provider and decouple them -- more so than what would be possible if they interacted directly (somebody once said - not without irony - that "any problem can be solved by another layer of indirection" - was it an R. Johnson?). This shields a service developer from the requirements of the consumer and allows her to focus on business logic -- rather than getting multiple stakeholders to agree on the non-functional aspects of the service interface.
ESBs place service virtualization "in the middle" - the MSE that Aaron Skonnard describes places it at the endpoint. Endpoint-based virtualization is becoming a new trend. WCF does it and some other containers - to a limited degree. Some LOB packaged applications start to offer this functionality at the endpoint (e.g., Siebel with the inclusion of Fusion Middleware). This is also in support of pure SOAP architectures that describe the enterprise as just a collection of SOAP-speaking endpoints (with reliability, security, etc. all being controlled through WS* protocols).
Virtualization at the endpoint, however, is not equivalent to virtualization in the middle. It limits you to those aspects of a communication that occur -- well, at the endpoint -- just before hitting the business logic. All aspects that are more centralized conceptually, are more difficult to address. Think about location transparency, for example. If this aspect is implemented on a central, shared intermediation layer it is simple to move backend service providers to new locations. If there is no such infrastructure, then every consumer must be able to directly address the provider endpoint. Either you configure these addresses in peer-to-peer fashion (which is a mess) or you do some kind of lookup: this could be via DNS (only works with HTTP), a registry or a centralized resolver service. The two latter solutions require that the consumer endpoint (either the stack or the business logic) handles the address resolution.
Then, think about content-based routing -- and things get more difficult. This would be a functionality that the consumer endpoint would provide -- or an intermediate infrastructure service. Other examples for virtualization/intermediation requirements taht map more easily onto a (conceptuall) centralized implementation, rather than onto a set of distributed smart endpoints:
- messaging patterns (pub/sub, reliable, async) - reliable queueing can be done at the endpoint but it's a maintenance nightmare (think trapped messages, poison messages, ...)
- complex event processing - for correlation/pattern detection, all events must be brought together at the event correlation engine for the application of rules
- message routing (addressing, CBR) - as mentioned above
- layered security defense - combination of multiple security mechanisms where threats are filtered out at multiple policy enforcement points (PEP) along the communication path
- control point for runtime governance (can you really rely on your distributed containers?) - to some degree, since you can monitor messages in progress, but not the availability of services that are not in use
- Provididing multiple interfaces to different clients at different network locations - maybe you only want to make a certain "engineered" interface (different operations, different policies) available to another department, DMZ-external, etc.

Friday, March 13, 2009
Developer Summit 2009 - Stockholm
I'll be speaking together with Herbjörn Wilhelmsen at the Developer Summit 2009 in Stockholm about How to Survive the Bermuda Triangle of Legacy, Services and ESB.Developer Summit, Scandinavia’s leading developer conference is turning 5 years! This year 40 sessions on 6 tracks, key-notes and workshops are planned.
Sunday, February 15, 2009
SOA Forum Schweiz 2009

The SOA Forum 2009 will take place May 12-13 in Zurich Sihlcity's cinema mediaplex. The motto this year: "Beyond the hype and theoretical discussions. Use SOA right – succeed in practice”.
Be sure to mark the date in your calendars! We'll have some really interesting and controversial keynote speakers at the start this year - wrapped up with a panel discussion.
- Thomas Erl: "Next Generation SOA"
- Anne Thomas Manes (Burton): “What Becomes of SOA During Though Economic Times?”
- Michael von Uechtritz (Gartner): final keynote
Wednesday, February 04, 2009
SOA-ESB book page & interview

Thanks to Thomas Erl for just setting up this brand-new page on the upcoming ESB Architecture for SOA book. There is an associated audio podcast on InformIT where Joe McKendrick interviews Mark Little and myself about some of the ESB-related patterns that made it into the SOA design patterns book.











