{"id":797,"date":"2011-07-20T16:48:26","date_gmt":"2011-07-20T23:48:26","guid":{"rendered":"http:\/\/www.visophyte.org\/blog\/?p=797"},"modified":"2011-09-28T19:22:49","modified_gmt":"2011-09-29T02:22:49","slug":"overview-ownershipcommunication-graphs-for-rich-execution-logs","status":"publish","type":"post","link":"https:\/\/www.visophyte.org\/blog\/2011\/07\/20\/overview-ownershipcommunication-graphs-for-rich-execution-logs\/","title":{"rendered":"overview ownership\/communication graphs for rich execution logs"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-798\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/relationship-overview-spgm.png\" alt=\"relationship-overview-spgm\" width=\"518\" height=\"444\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/relationship-overview-spgm.png 518w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/relationship-overview-spgm-300x257.png 300w\" sizes=\"auto, (max-width: 518px) 100vw, 518px\" \/><\/p>\n<p>My <a href=\"http:\/\/www.visophyte.org\/blog\/2011\/07\/12\/new-adventures-in-rich-execution-logs-for-debugging-and-program-understanding\/\">last blog post<\/a> covered the fancy pants logging being used for the deuxdrop project. \u00a0To summarize:<\/p>\n<ul>\n<li>we have loggers<\/li>\n<li>they are organized by ownership hierarchy<\/li>\n<li>they provide sufficient metadata that we can reconstruct the loggers that were talking to each other<\/li>\n<\/ul>\n<div>This can result in a lot of loggers. \u00a0If you look at the <a href=\"https:\/\/clicky.visophyte.org\/examples\/arbpl-loggest\/20110712\/\">log output from the last blog post<\/a> (caution: BIG JSON file in standalone ArbPL instance) you might notice a list of loggers that looks something like the below, except not quite as sideways or shrunk:<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-799\" title=\"\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/prev-rev-logger-hier.png\" alt=\"last generation's logger hierarchy rotated sideways\" width=\"600\" height=\"196\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/prev-rev-logger-hier.png 600w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/prev-rev-logger-hier-300x98.png 300w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>Obviously, that much raw data is not super helpful. \u00a0So now we process the hierarchy, constructing graph nodes or aggregate graph nodes for things that are interesting in their own or in aggregate. \u00a0We use <a href=\"http:\/\/mbostock.github.com\/d3\/\">d3<\/a>\u00a0of the vaunted <a href=\"http:\/\/mbostock.github.com\/protovis\/\">protovis<\/a> lineage\u00a0to visualize the network and\u00a0<a href=\"http:\/\/www.graphviz.org\/\">graphviz<\/a> to lay it out.<\/p>\n<p>d3 has a super-polished, fun-to-use interactive force-directed graph implementation, but graphviz&#8217;s circo layout produces better results. \u00a0Given that <a href=\"https:\/\/github.com\/asutherland\/arbitrarypushlog\">ArbPL<\/a>\u00a0already has a processing step for intake, it wasn&#8217;t too much extra work to include a step where we use the same JS code as on the client to generate a dot file, pass it to circo to lay it out, then extract the layout information and store it with the log data. \u00a0Many props to\u00a0<a href=\"http:\/\/algorithmique.net\/\">Gregoire Lejeune<\/a> for his <a href=\"https:\/\/github.com\/glejeune\/node-graphviz\">node-graphviz<\/a> bindings that made it so easy to do.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-800\" title=\"\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/overview-z-joined-notif-highlighted.png\" alt=\"overview-z-joined-notif-highlighted\" width=\"517\" height=\"445\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/overview-z-joined-notif-highlighted.png 517w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/overview-z-joined-notif-highlighted-300x258.png 300w\" sizes=\"auto, (max-width: 517px) 100vw, 517px\" \/><\/p>\n<p>Although the overview graph on its own is neat, it becomes useful by showing us the involved loggers\/actors in a test step by highlighting them. \u00a0It does this by listening for <a href=\"https:\/\/github.com\/asutherland\/wmsy\">wmsy<\/a>&#8216;s focus change events to know what the focused test step is. \u00a0(Since mouse interaction also causes toggling, it&#8217;s easiest to appreciate the process by using the up\/down arrow keys to change focus without expanding the steps. \u00a0And if you want to toggle the expanded state without involving the mouse, you can hit the enter key.)<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-801\" title=\"\" src=\"http:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/z-joined-step-callout-causes-highlighting.png\" alt=\"z-joined-step-callout-causes-highlighting\" width=\"624\" height=\"107\" srcset=\"https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/z-joined-step-callout-causes-highlighting.png 624w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/z-joined-step-callout-causes-highlighting-600x102.png 600w, https:\/\/www.visophyte.org\/blog\/wp-content\/uploads\/2011\/07\/z-joined-step-callout-causes-highlighting-300x51.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/p>\n<p>For reference, the steps look like the above, and the highlighted one has the ever-so-fashionable glowing fancy focus ring.<\/p>\n<p>If you would like to see this for yourself, you can check out the archived (and therefore less likely to break) standalone version of the example log file:\u00a0<a href=\"https:\/\/clicky.visophyte.org\/examples\/arbpl-loggest\/20110720\/\">https:\/\/clicky.visophyte.org\/examples\/arbpl-loggest\/20110720\/<\/a>. \u00a0The previously super-huge JSON file is now about an order-of-magnitude smaller because I started eliding large strings that are of no human interest.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>My last blog post covered the fancy pants logging being used for the deuxdrop project. \u00a0To summarize: we have loggers they are organized by ownership hierarchy they provide sufficient metadata that we can reconstruct the loggers that were talking to &hellip; <a href=\"https:\/\/www.visophyte.org\/blog\/2011\/07\/20\/overview-ownershipcommunication-graphs-for-rich-execution-logs\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[3,8,4],"tags":[106,108],"class_list":["post-797","post","type-post","status-publish","format-standard","hentry","category-mozilla","category-program-execution","category-visualizing","tag-arbpl","tag-loggest"],"_links":{"self":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/797","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/comments?post=797"}],"version-history":[{"count":8,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/797\/revisions"}],"predecessor-version":[{"id":826,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/posts\/797\/revisions\/826"}],"wp:attachment":[{"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/media?parent=797"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/categories?post=797"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.visophyte.org\/blog\/wp-json\/wp\/v2\/tags?post=797"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}