stream http://www.roland-stuehmer.de/tags/stream en Immutability and Event Derivation in RDF http://www.roland-stuehmer.de/content/immutability-and-event-derivation-rdf <span property="schema:name" class="field field-node--title field-name-title field-type-string field-label-hidden">Immutability and Event Derivation in RDF</span> <span rel="schema:author" class="field field-node--uid field-name-uid field-type-entity-reference field-label-hidden"><a title="View user profile." href="/users/rolandstuehmer" lang="" about="/users/rolandstuehmer" typeof="schema:Person" property="schema:name" datatype="" class="username">roland.stuehmer</a></span> <span property="schema:dateCreated" content="2014-10-01T14:54:13+00:00" class="field field-node--created field-name-created field-type-created field-label-hidden">Wed, 10/01/2014 - 16:54</span> <div property="schema:text" class="clearfix field field-node--body field-name-body field-type-text-with-summary field-label-hidden"> <div class="field-items"> <div property="schema:text" class="field-item"><p>"In many event processing systems [...] events are immutable"<sup id="fnref:Luckham and Schulte 2011"><a href="#fn:Luckham and Schulte 2011" class="footnote-ref">1</a></sup>. This stems from the definition of what an event is: "An event is an occurrence within a particular system or domain; it is something that has happened, or is contemplated as having happened [...]"<sup id="fnref:Etzion and Niblett 2010"><a href="#fn:Etzion and Niblett 2010" class="footnote-ref">2</a></sup>. So events cannot be made to unhappen.</p> <p><strong>Open Question:</strong> Does this apply to all systems/applications/usecases or just to "many" as stated above?</p> <p>I made immutability a <strong>general assumption</strong> in my work<sup id="fnref:Stühmer 2014"><a href="#fn:Stühmer 2014" class="footnote-ref">3</a></sup>. It is very useful for building systems (distributed systems, consistency, ...).</p> <p><strong>Q: How can a Stream processing agent process events if they are immutable?</strong></p> <p><strong>A:</strong> Every processing task produces new <em>derived events</em> as results. Advantage: the underived events are still available for other uses and remain immutable.</p> <p>For <abbr title="RDF Stream Processing">RSP</abbr> this means: (1) create a new (unique) graph for the derived event (2) possibly link back to the base event(s) thus enabling drill-down or root cause / provenance analysis of the derived event. The links can be made with <code>DUL:hasConstituent</code> from DOLCE Ultralight<sup id="fnref:Gangemi 2009"><a href="#fn:Gangemi 2009" class="footnote-ref">4</a></sup>. In my own work<sup id="fnref:Harth and Stühmer 2011"><a href="#fn:Harth and Stühmer 2011" class="footnote-ref">5</a></sup> I use a new <code>:members</code> property to link from a derived event to its simple events. The property is a subproperty of the mentioned <code>DUL:hasConstituent</code>.</p> <p><strong>Observation:</strong> We talk about adding "received time" and other metadata later by receiving agents: Adding triples later to the event graph with graphname as subject can still be legal and considered as <em>amending</em> the event header. Much like with email: headers can be added by intermediate mail servers but the mail body and ID are immutable.</p> <div class="footnotes"> <hr /> <ol> <li id="fn:Luckham and Schulte 2011"> <p>Luckham, D. C. &amp; Schulte, R. <a href="http://www.complexevents.com/2011/08/23/event-processing-glossary-version-2-0/">Event Processing Glossary - Version 2.0 (2011)</a>&#160;<a href="#fnref:Luckham and Schulte 2011" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Etzion and Niblett 2010"> <p>Etzion, O. &amp; Niblett, P. Event Processing in Action Manning Publications Co. (2010)&#160;<a href="#fnref:Etzion and Niblett 2010" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Stühmer 2014"> <p>Stühmer, R. <a href="http://dx.doi.org/10.5445/KSP/1000043122">Web-oriented Event Processing Karlsruhe Institute of Technology, KIT Scientific Publishing, Karlsruhe, 2014</a>&#160;<a href="#fnref:Stühmer 2014" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Gangemi 2009"> <p>Gangemi, A. <a href="http://ontologydesignpatterns.org/wiki/Ontology:DOLCE+DnS_Ultralite">DOLCE+DnS Ultralite (<abbr title="DOLCE+DnS Ultralite">DUL</abbr>) (2009)</a>&#160;<a href="#fnref:Gangemi 2009" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Harth and Stühmer 2011"> <p>Harth, A. &amp; Stühmer, R. <a href="http://km.aifb.kit.edu/sites/lodstream/">Publishing Event Streams as Linked Data Karlsruhe Institute of Technology, FZI Forschungszentrum Informatik (2011)</a>&#160;<a href="#fnref:Harth and Stühmer 2011" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> </ol> </div> </div> </div> </div> <section class="field field-node--comment-node-article field-name-comment-node-article field-type-comment field-label-hidden comment-wrapper"> </section> <div class="field field-node--field-tags field-name-field-tags field-type-entity-reference field-label-inline clearfix"> <div class="field-label">Tags</div> <div class="field-items"> <div class="field-item"><a href="/tags/immutability" property="schema:about" hreflang="en">immutability</a></div> <div class="field-item"><a href="/tags/rdf" property="schema:about" hreflang="en">RDF</a></div> <div class="field-item"><a href="/tags/rsp" property="schema:about" hreflang="en">RSP</a></div> <div class="field-item"><a href="/tags/stream" property="schema:about" hreflang="en">stream</a></div> <div class="field-item"><a href="/tags/event-derivation" property="schema:about" hreflang="en">event derivation</a></div> </div> </div> Wed, 01 Oct 2014 14:54:13 +0000 roland.stuehmer 39 at http://www.roland-stuehmer.de/drupal-8 Stream Punctuation and RDF Stream Processing http://www.roland-stuehmer.de/content/stream-punctuation-and-rdf-stream-processing <span property="schema:name" class="field field-node--title field-name-title field-type-string field-label-hidden">Stream Punctuation and RDF Stream Processing</span> <span rel="schema:author" class="field field-node--uid field-name-uid field-type-entity-reference field-label-hidden"><a title="View user profile." href="/users/rolandstuehmer" lang="" about="/users/rolandstuehmer" typeof="schema:Person" property="schema:name" datatype="" class="username">roland.stuehmer</a></span> <span property="schema:dateCreated" content="2014-10-01T14:44:19+00:00" class="field field-node--created field-name-created field-type-created field-label-hidden">Wed, 10/01/2014 - 16:44</span> <div property="schema:text" class="clearfix field field-node--body field-name-body field-type-text-with-summary field-label-hidden"> <div class="field-items"> <div property="schema:text" class="field-item"><p>Definition by Tucker et al.<sup id="fnref:Tucker et al. 2003"><a href="#fn:Tucker et al. 2003" class="footnote-ref">1</a></sup> and Maier et al.<sup id="fnref:Maier et al. 2005"><a href="#fn:Maier et al. 2005" class="footnote-ref">2</a></sup>:</p> <blockquote> <p>''A punctuation is a pattern <code>p</code> inserted into the data stream with the meaning that no data item <code>i</code> matching <code>p</code> will occur further on in the stream.''</p> </blockquote> <p>For event processing systems, events are the fundamental unit of information<sup id="fnref:Gupta and Jain 2011"><a href="#fn:Gupta and Jain 2011" class="footnote-ref">3</a></sup>. This means each event is processed atomically, i.e. completely or not at all. For <abbr title="Resource Description Format">RDF</abbr> stream processing systems this can cause problems if events are modelled as graphs consisting of multiple quadruples: How can a receiver of an event know that all quadruples pertaining to the event are transmitted in order to start processing the event?</p> <p>For streams of <abbr title="Resource Description Format">RDF</abbr> graphs punctuation can be used like this: A punctuation is a pattern ''p'' inserted into the quadruple stream with the meaning that no quadruples <code>i</code> from graph <code>p</code> will occur further on in the stream.</p> <p>Punctuation could be implemented using special ("magic") quadruples but when using the Web stack(!) we can do punctuation out-of-band, i.e. implement punctuation on a lower layer of the stack. For example, we can communicate through ''chunked transfer encoding'' (Fielding et al. 1999, Section 3.6.1)<sup id="fnref:Fielding et al. 1999"><a href="#fn:Fielding et al. 1999" class="footnote-ref">4</a></sup> from HTTP 1.1. Each chunk contains a complete graph and the receiver will know that after a chunk is received the event is completely received and can be processed further in an atomic fashion. There is a guarantee that no quads for this graph will arrive later. Using HTTP chunked connections no special (or magic) quads are needed.</p> <p>''Chunked transfer encoding'' is also used by the <abbr title="Resource Description Format">RDF</abbr> publish/subscribe middleware Ztreamy<sup id="fnref:Fisteus et al. 2014"><a href="#fn:Fisteus et al. 2014" class="footnote-ref">5</a></sup> to provide long-lived connections using pure HTTP with the goal of disseminating events to subscribers. Further related work<sup id="fnref:Shinavier 2010"><a href="#fn:Shinavier 2010" class="footnote-ref">6</a></sup> investigates the exchange of <abbr title="Resource Description Format">RDF</abbr> over different protocols such as <abbr title="Extensible Messaging and Presence Protocol">XMPP</abbr> on top of HTTP (and thus <abbr title="Transmission Control Protocol">TCP</abbr>) but even <abbr title="User Datagram Protocol">UDP</abbr>. However, none of these protocols provides pure HTTP stream URIs which are easily referenced in Linked Data.</p> <div class="footnotes"> <hr /> <ol> <li id="fn:Tucker et al. 2003"> <p>Tucker, P.; Maier, D.; Sheard, T. &amp; Fegaras, L. Exploiting punctuation semantics in continuous data streams Knowledge and Data Engineering, IEEE Transactions on, 2003, 15, 555-568 [http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1198390]&#160;<a href="#fnref:Tucker et al. 2003" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Maier et al. 2005"> <p>Maier, D.; Li, J.; Tucker, P.; Tufte, K. &amp; Papadimos, V. Semantics of Data Streams and Operators Proceedings of the 10th International Conference on Database Theory, Springer-Verlag, 2005, 37-52 [http://datalab.cs.pdx.edu/niagaraST/icdt05.pdf]&#160;<a href="#fnref:Maier et al. 2005" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Gupta and Jain 2011"> <p>Gupta, A. &amp; Jain, R. Managing Event Information: Modeling, Retrieval, and Applications Managing Event Information, Morgan &amp; Claypool Publishers, 2011&#160;<a href="#fnref:Gupta and Jain 2011" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Fielding et al. 1999"> <p>Fielding, R.; Gettys, J.; Mogul, J.; Frystyk, H.; Masinter, L.; Leach, P. &amp; Berners-Lee, T. Hypertext Transfer Protocol -- HTTP/1.1 RFC Editor, 1999 [http://www.w3.org/Protocols/rfc2616/rfc2616.html]&#160;<a href="#fnref:Fielding et al. 1999" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Fisteus et al. 2014"> <p>Fisteus, J. A.; García, N. F.; Fernández, L. S. &amp; Fuentes-Lorenzo, D. (2014), 'Ztreamy: A middleware for publishing semantic streams on the Web ', Web Semantics: Science, Services and Agents on the World Wide Web 25(0), 16 - 23.&#160;<a href="#fnref:Fisteus et al. 2014" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> <li id="fn:Shinavier 2010"> <p>Shinavier, J. Optimizing real-time <abbr title="Resource Description Format">RDF</abbr> data streams CoRR, 2010, abs/1011.3595 [http://arxiv.org/abs/1011.3595]&#160;<a href="#fnref:Shinavier 2010" class="footnote-backref">&#8617;&#xFE0E;</a></p> </li> </ol> </div> </div> </div> </div> <section class="field field-node--comment-node-article field-name-comment-node-article field-type-comment field-label-hidden comment-wrapper"> </section> <div class="field field-node--field-tags field-name-field-tags field-type-entity-reference field-label-inline clearfix"> <div class="field-label">Tags</div> <div class="field-items"> <div class="field-item"><a href="/tags/stream" property="schema:about" hreflang="en">stream</a></div> <div class="field-item"><a href="/tags/punctuation" property="schema:about" hreflang="en">punctuation</a></div> <div class="field-item"><a href="/tags/rdf" property="schema:about" hreflang="en">RDF</a></div> <div class="field-item"><a href="/tags/rsp" property="schema:about" hreflang="en">RSP</a></div> <div class="field-item"><a href="/tags/http" property="schema:about" hreflang="en">HTTP</a></div> <div class="field-item"><a href="/tags/chunked-transfer-encoding" property="schema:about" hreflang="en">chunked transfer encoding</a></div> </div> </div> Wed, 01 Oct 2014 14:44:19 +0000 roland.stuehmer 38 at http://www.roland-stuehmer.de/drupal-8