Complex Event Processing

https://en.wikipedia.org/wiki/Complex_event_processing

Event processing is a method of tracking and analyzing (processing) streams of information (data) about things that happen (events),^[1] and deriving a conclusion from them. Complex event processing, or CEP, is event processing that combines data from multiple sources^[2] to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats)^[3] and respond to them as quickly as possible.

These events may be happening across the various layers of an organization as sales leads, orders or customer service calls. Or, they may be news items,^[4] text messages, social media posts, stock market feeds, traffic reports, weather reports, or other kinds of data.^[1] An event may also be defined as a "change of state," when a measurement exceeds a predefined threshold of time, temperature, or other value. Analysts suggest that CEP will give organizations a new way to analyze patterns in real-time and help the business side communicate better with IT and service departments.^[5]

The vast amount of information available about events is sometimes referred to as the event cloud.^[1]

Notable vendors and products[edit]

• SQLstream SQLstream’s stream processing platform, s-Server, provides a relational stream computing platform for analyzing large volumes of service, sensor and machine and log file data in real-time.
• Microsoft StreamInsight Microsoft CEP Engine implementation ^[20]
• openPDC — A set of applications for processing streaming time-series data in real-time.
• Apama - A Complex Event Processing Platform that monitors rapidly moving event streams, detects and analyzes important patterns, and takes action according to rules.^[21]
• StreamBase Systems - A visual development platform and high performance event server for rapidly building and deploying real-time event-based applications. - Now Owned by TIBCO Software
• SAP ESP - A low-latency, rapid development and deployment platform that allows processing multiple streams of data in real time ^[22]
• TIBCO BusinessEvents & Streambase - CEP platform and High Performance Low Latency Event Stream Processing
• WebSphere Business Events
• Informatica RulePoint - High performant, scalable solution by Informatica.
• Drools Fusion
• GigaSpaces XAP
• Oracle Event Processing - A solution for building applications to filter, correlate, and process events in real time.
• Esper Complex event processing for Java and C#.
• Siddhi Complex event processing written in Java. Designed as part of a series of middleware components.

https://www.youtube.com/watch?v=UWq7uSXaoz8 내용중

제품

상용제품

TIBCO : TIBCO BusinessEvents : 업계 최고 제품

StreamBase Systems : StreamBase CEP : 정교한 CEP 제품

SAP : Sybase Event Stream Processor : Sysbase 인수

IBM : InfoSphere Streams : 테러 방지 목적으로 시작 멀티미디어 스트림 프로세싱 가능

Microsoft : StreamInsight : 

Informatica : RulePoint

오픈소스

Esper

quick start project https://github.com/tedwon/cep-esper-quick-start

입력 (이벤트를 수집) 어뎁터를 만들어야 함

처리 (실시간 처리) 

결과 (다양한 이벤트의 복합 영향도 패턴 인식 등을 구현할 수 있음)

EPL SQL 형식의 Event 처리 스크립트 언어

• Event filtering 
• Sliding data windows and aggregation
• grouped windows and output rate limiting
• joins and outer joins
• subquery
• historical or reference data

Event Streams 에서 EPL Statements 를 이용해 데이터를 검색할 수 있음

join 은 RDBMS 나 Apache HBase UDF 와 할 수 있다. 

Example

from pattern [a -> (b and c)]

b c 이벤트가 발생하고 a 가 발생하면 알람을 달라 라고 event 의 correlation 을 정할 수 있음 

개발전략

모니터링 이벤트 정의

이벤트 타입(포멧) 정의

EPL 정의

output adapter 개발 등록

input adapter 개발 등록

CEP 개발 주요 포인트

Runtime 환경 결정

Zero-downtime 지원 여부 결정

Runtime 시 EPL 업데이트 지원 여부 결정

Event 타입과 용량 결정 

솔루션 만들고 JBOSS RHQ 로 모니터링