CloudI Presentation at Chicago Erlang User Group

I recently gave a talk on the CloudI integration framework at the May 2015 meeting of the Chicago Erlang User Group.  The presentation is available here

CloudI

CloudI is a relatively new open source project for developing cloud-based solutions.  The main project web site is www.cloudi.org.  According to the project's web site:

CloudI is an Integration Cloud that focuses on flexible integration that is efficient, scalable, and fault-tolerant. CloudI does not force a user to use particular software libraries but instead provides light-weight interfaces for integration. Scalability and fault-tolerance are both provided by CloudI's usage of the Erlang programming language. This means that no Hypervisor or commercial service is necessary to make CloudI's processes reliable, so there can be a performance benefit when using CloudI.

A few years ago, I developed a book-recommendation application that served as a test bed for experimenting with recommendation engines.  I built the system using PHP, MySQL, the JQuery HTML library, and an open-source recommendation engine written in .Net.  I am planning to rewrite the application to use the CloudI environment and a more service-oriented approach. There do not appear to be a lot of examples on the practical use of CloudI, so I am planning on documenting my experience in a series of detailed tutorials that will hopefully add up to a good overall case study.

Elements of UML Style

Recently I've been developing a large number of UML diagrams that are part of a Solution Architecture Document. I ran across a great little book that has helped me improve the diagrams style, consistency, and make them easier to understand.

The book is The Elements of UML 2.0 Style by Scott W. Ambler.   The book assumes that you understand basic UML techniques and modeling already.  It provides a set of standards and rules for different types of diagrams along with a justification.

The book is inexpensive and easy to use as a basic reference.  Highly recommended.

Esper Complex Event Processor

I have recently been experimenting with an open source product called Esper that provides a complex event processor (CEP) engine.   Commercial support is provided by EsperTech along with an enterprise edition that offers some additional capabilities.

An article from TheServerSide describes Esper this way:

"The Esper engine works somewhat like a database turned upside-down. Instead of storing the data and running queries against stored data, Esper enables applications to store queries (or statements) and run the data through them. Response from the Esper engine is thus real-time when conditions occur that match queries. The execution model is thus continuous rather than only when a query is submitted."

I used  Esper to build an application that monitors an Oracle SOA Suite for Healthcare Integration cluster and provides information about the status of endpoints and different views of the messages flowing across the cluster.  I had done something similar using Splunk earlier in the year and wanted to see how Esper would compare.

I started by creating simple Java POJO classes that represented the events of interest.  For example, the class for an HL7 message looks like this:

public class MessageEvent {
    String id;
    String direction;
    String messageType;
    String senderName;
    String receiverName;
    String state;
    String docTypeName;
    String docProtocolName;
    String docProtocolVersion;
    GregorianCalendar eventDate;
           ....

Next, I defined various windows using Esper's Event Processing Language (EPL).  A window is a logical view of a sliding time scale.  Using a window makes queries easier to write.  For example, to create a window that retains messages for the last 15 minutes, the definition looks like:

create window Messages15mWindow.win:time(15 minutes) as 
select * from MessageEvent

Finally, I created queries that retrieved data of interest from these windows.  For example, to retrieve the number of messages that have appeared in the last 15 minutes, the query looks like:

select count(*) as quantity, direction, senderName, receiverName, state, docTypeName 
from Messages15mWindow 
group by direction, senderName, receiverName, state, docTypeName

I found the Esper EPL syntax to be very easy to use and very familiar if you have used SQL before.
The queries listed above are registered with the Esper engine and when new events arrive in the system, Esper will notify listeners that new results are waiting to be processed.  In my application, I used a reverse-ajax library named Direct Web Remoting (DWR) to broadcast the results to a web page.  Here's an example:

Configuration Management Database (CMDB)

On many software development or implementation projects there is a need to track important work products, components, and their relationships.  Many mid-size companies use Microsoft SharePoint for this, but SharePoint has some issues with data integration and usability.

I've been using an open-source tool named that CMDBuild that I highly recommend.  You can read more about it and get the software from http://www.cmdbuild.org

It is very simple to setup new data structures for the important stuff you want to track and define relationships.  CMDBuild also has great support for developing custom reports using Jasper Reports and support for workflow management.  It also includes a SOA interface for data integration with other systems as well as an import/export utility.

CMDBuild uses a PostgresSQL database to store all data and history of changes so you can always work at the database level for advanced functionality if it is not supported from the CMDBuild web application.

Software Selection Methodology

I recently worked with a client to do a detailed open-source software assessment and selection.  I found a great methodology and set of tools specifically designed for open-source software located at  http://www.qsos.org/

 The QSOS tools support the development of a "template" that defines the selection criteria that is appropriate for the product you are considering.  The criteria are defined using an XML file.
For example, a criterion for training is shown below.

<element name="services" title="Services">
    <desc>Services offering</desc>
    <element name="training" title="Training">
       <desc0>No offer of training identified</desc0>
       <desc1>Offer exists but is restricted geographically and to one
              language or is provided by a single contractor</desc1>
       <desc2>Rich offers provided by several contractors, in serveral
              languages and split into modules of gradual levels</desc2>
       <score></score>
       <comment></comment>
    </element>
    
    <!-- More elements go here -->
</element>

Once the selection criteria are defined, QSOS provides a great tool for evaluating a specific product using the criteria.  This tool uses XUL standards and can run within a web-browser or as a standalone tool.  A screenshot of the tool running the criteria shown in XML is listed below.




The tool can also display the scores using a radar-type plot that can show areas of weakness or strength for a particular product. 




The QSOS web site listed above contains evaluations that other people have done that might be directly useful for your needs.