Bricodash

From Hack Manhattan Wiki
Revision as of 03:21, 12 April 2019 by Beadsland (Talk | contribs) (Building Calendar)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Bricodash screenshot

Github icon.jpg Github Repository | Beads Land-Trujillo.jpg A Beadsland Creation

An idlying chromecast is a bad chromecast. Cast a dashboard instead.

Bricodash is an informational dashboard displayed on an overhead TV in the main work area and on a small monitor mounted by the coat-racks next to the machine shop.

The server back-end is housed on a black tower named bricolage—literally, "Something constructed using whatever was available at the time".

See also:

 

Overview

In October of 2017, Hack Manhattan installed a TV by the workspace tables. To make it easily accessible to everyone in the space, we added a Chromecast. To make that more useful, we're casting a "default" dashboard now. The same default dashboard is also displayed on a screen over the intercom at the entrance to our shop. This screen is driven by an old Raspberry Pi.

Bricodash represents a major revision to last year's defaultcast, originally developed by Michael, providing more functionality while using a thin-client architecture.

If the early version Raspberry Pi we're using is low powered, the Chromecast is especially so. The goal has been to provide as much functionality as possible while keeping client load to a minimum. With this in mind, we rely heavily on a server-side cron hierarchy to update various HTML components, allowing the browser to poll for those files at appropriate intervals.

At last count, Bricodash incorporates eight (8) different APIs.

Design Philosophy

I'm an old-school Web developer, and prefer to avoid bloat and wall-of-text source files, in favor of clean, single-purpose, Small-is-Beautiful modules that do what they do with minimal byte-tonage. That said, the source files hacked together to get the system off the ground are just that, hacks. They're ugly and experimental and will hurt the eyes and brains of anyone accustomed to the regimented, framework-heavy best practices of contemporary Web design. Hic sunt dracones.

This is not meant to be an example of professional, full-stack Web development. It is meant, instead, to demonstrate what can be accomplished when working without a net, allowing creativity and resourcefulness to drive the writing process, rather than convention and established practices. As when an author writes a book, the editing process is a necessary step, but should not come before, let alone at the expense of, the completion of a first draft.

Error Handling

Each panel polls HTML files built periodically by back-end scripts on the bricodash server. Each file includes a timestamp tag indicating when the file was last generated by its respective script. If a script fails to complete successfully, the client will merely receive the last successfully written version when the file is next polled.

As part of the polling step, each panel checks the age of the HTML file received against its polling interval. When age exceeds polling interval by a small multiple, the file is deemed stale and a yellow warning icon (⚠️) is displayed in the center of the panel. Intermittent appearances of yellow warning icons can indicate network issues or excessively short polling cycles. Persistent indicators are a sign that the relevant back-end script should be executed at the console to determine why it might be failing.

Calendar Errors

The community and building calendars are compiled via a two-stage process. For each calendar data source, there is a back-end script that polls that source at an interval appropriate to its API and/or volatility. The results of these polls are written to json files for later use. A separate, master script then collates the contents of these variously updated JSON files to produce consolidated HTML files for display on the dashboard.

The master script does not check for freshness of the intermediate JSON files. Rather, each successful run of the master script will show as having a fresh timestamp, regardless of the age of the source JSON files. No yellow warning icon will be shown if any of the data source scripts fail.

Instead, a failing source script will become apparent if either the calendar is seen to be listing events that have passed days prior and/or fails to show current event listings that one would expect from the source. In either case, an error condition can then be confirmed by checking the system timestamps on all intermediate calendar JSON files.

(It should be noted that there are some events, notably events that start at a specific time, that may continue to be listed on the day immediately following their start time. Usually these will be holidays on lunar or lunisolar calendars, astronomical events, or events that last beyond midnight. Such yesterday events do not represent errors or script failures, but rather are by design.)

Pending Tasks

  • calendars (short term)
    • revise primaries calendar event listings to not appear on off years
    • castles: icalevents: ValueError: RRULE UNTIL values must be specified in UTC when DTSTART is timezone-aware
    • correctly calculate date of International Astronomy Day (email out to Astronomical League for better calculation rules)
    • confirm names on shared birthdays are randomized
  • reliability
    • add decoration to indicate when a post is from a thread rather than from main channel
    • add a webhook to ping #netops when camera feed is down
    • enable SSL for camera feeds (requested for full conversion to new sous veil system)
    • find new sweet spot for camera throttling given new network conditions
    • replace failing microSD on pishop (and restart daily cron reboot)
    • add route to trigger a pishop browser cache flush
  • documentation
  • cloud log
    • add boxrs-based git repositories to Cloud Log panel
    • add remaining sources to HM Cloud Log panel
  • calendars
    • refactor brite search by location routine to common library
    • add nyc resistor, fat cat fab lab, and makerbar to Community Calendar
    • finish fleshing out remainder of movable feasts for Community Calendar
    • enable toggling between Welcome header and holiday greetings ^
    • discard low priority "Holiday" events when today is overcrowded
  • cleanup
    • integrate citymapper API for local subway stations
    • refactor source files from initial launch
    • squish and push fork to upstream
    • migrate from bricolage to less power-hungry hardware
    • reimplement bricolage to run as embedded system under elixir/scenic/nerves
  • explore future development ideas

 

Panels

The dashboard is comprised of five panels, or panes, and two overlays, described below starting clockwise from the top:

Door Cam

A cleaned up stream from our door camera graces the upper-right corner of the dashboard. The dashboard pulls an MJPG stream from a proxy gateway. Rather than relay the MJPG stream raw to our display devices, the gateway requests snapshots from the device hosting the door camera, with a slight sleep between requests, to slow down the frame rate. This is done to avoid stalling out the Chromecast and Pi devices. Additionally, our script discards broken frames to reduce the number of grey-bottom artefacts appearing in the displayed feed.

We've recently replaced the router that was serving bricolage and our Chromecast to better equipment. We've witnessed at least one lock-up of the Chromecast since then. Suspected cause is a higher frame rate being fed by the gateway in the absence of broken frames to discard. Going to have to experiment with the sleep duration between requests in light of the better throughput.

To be clear, a Chromecast seems perfectly capable of handling a full speed MJPG stream if that's all it's being asked to do. Once that MJPG feed begins sharing cycles with Javascript setInterval triggers, things get less stable. The more the Chromecast is asked to repeatedly do in addition to displaying the stream, the more that stream starts to look like a firehose. Eventually, the device freezes, becoming unresponsive to requests to stop the dashboard or start playing other content, and must by powercycled to get working again. Lowering the frame rate below full stream allows us to both stream a camera feed and actively update other content on the dashboard without overtaxing the Chromecast.

The upper-right corner of the feed, in turn, displays regularly updated weather information, including wind speed and "feels like" temperatures, when appropriate. A time clock, meanwhile, is superimposed over the lower-right corner of the camera feed.

Future Ideas

Subway Status

The lower-right corner of the dashboard displays subway line status information, as provided by our local transit authority. The widget is a variation of that intermittently displayed on the digital information kiosks now becoming common in the city.

As provided, however, the transit authority's widget provides only for real-time status at time of page load. Thus, our back-end polls the widget, and caches the result for polled retrieval via a gateway script. This allows for any number of dashboard display devices to show minute-by-minute updates to subway status while keeping traffic to the information provider constant. Additionally, our back-end script strips live components unnecessary to a non-interactive display and munges colors in styles and images to conform to the rest of the dashboard.

That all said, the script providing this functionality is currently an imperative hack, and could do with some refactoring.

Future Ideas: The polling module is ripe for refactoring.

Slack Channel

Centered on the bottom half of the dashboard are recent posts to the #hackerspace channel on Hack Manhattan's Slack. This panel does not attempt to emulate the content layout of a Slack client. Instead, various strategies are used to concisely and crisply represent the content of the channel while making optimal use of the limited screen real estate available.

Originally implemented via Slack's deprecated "Integration" mechanism, the back-end script for this panel has recently been migrated to a Slack's newer "App" model. The new model implements a finagley permissions system consisting of scopes and paired API keys (one for the app proper, one for a bot associated with the app), where certain methods available to one API context are not available to the other.

At this time, it appears that the migration was successful. But it is possible that certain obscure features not regularly used will, when introduced into the channel stream, result in a missing_scope exception that will temporarily crash the back-end script, resulting in the display of yellow warning icon (⚠️) within the panel.

Future Ideas: Draft a Privacy Policy

HM Cloud Log

Hm cloud log logogram.png
In the lower-left corner of the dashboard is a panel with a logographic heading that translates as "Hack Manhattan Cloud Write Scroll", i.e., "Hack Manhattan Cloud Log".

This panel provides a running log of recent changes to the Hack Manhattan community's online activities. This includes changes to pages on this mediawiki server and to event listings on our Meetup group, as well as a activity on Hack Manhattan's Github repository. In addition to listing recent activity for all projects under the the github.com/hackmanhattan, members may optionally register their own repository services accounts via Bricodash/Extra:Repos.

Meanwhile, a thumbnail gallery feature indicates whenever new images are uploaded to this mediawiki server or our Meetup group. (Integration of our Instragram and Twitter accounts is pending.)

Future Ideas

Event Calendars

The upper-left hand side of the dashboard alternates between a calendar showing events of interest to the Hack Manhattan community and a calendar showing events occurring within the Rat Park building.

Event listings are polled periodically from a number of different cloud services and data sources, each by way a source-specific script that extracts relevant event information then writes formatted and decorated output to an intermediary JSON file. A master script then regularly collates these files to generate three files, two HTML files for polling by the dashboard, and a third file formatted for posting to Slack.

Community Calendar

The community calendar draws events from the following sources:

  • Meetup
    • Hack Manhattan — Meetup Group
    • Math for Math's Sake — Meetup Group (co-hosted by Hack Manhattan)
    • Hacktoberfest — general event search (annual festivities around town)
  • Google Calendar
    • Babycastles — neighboring organization (with ties to our community)
    • Hack Manhattan — private events at space (not listed on Meetup)
  • Local Text Files
    • birth.list — birthdays of notable makers, hackers, cultural figures and scientists
    • geek.cal — various holidays mathy, sciencey, silly and makery
    • local.cal — recurring events of regional interest
    • usnat.cal — national public holidays and related events
    • trivia.cal — tech anniversaries and natural phenomena
  • multi.py — module for calculating multi-day, lunar, lunisolar and astronomical events (STILL UNDER DEVELOPMENT)
  • Bricodash/Extra:Calendar — member-supplied events of interest

Without line wrapping, the calendar panel has room for listing seven (7) upcoming events on the main screen. The smaller screen by the shop can display eight (8) lines of events, but as lines are shorter, lines wrap more often. To privilege events happening at Hack Manhattan's space, the number of non-space events are capped at four (4). Within this constrained line count, more trivial sources are further filtered to only appear on the day of or immediately before, to avoid (somewhat) crowding out of more salient upcoming events like national holidays.

Future Ideas

Building Calendar

Although relevant to our community, the Rat Park calendar is not of primary interest. For this reason, the Building Calendar is displayed only one out of every five minutes, just long enough to get a general glance of events happening in the building.

The event calendar draws events from the following sources:

  • Meetup
    • Hack Manhattan — Meetup Group
    • Math for Math's Sake — Meetup Group (co-hosted by Hack Manhattan)
    • 137 W 14th Street — general search (non-HM events hosted in our building)
  • Google Calendar
    • Hack Manhattan — private events at space (not listed on Meetup)
  • Eventbrite
    • Secret Loft — DIY event space that shares our floor
    • Offside Tavern — sports bar on building's ground floor
  • autogenerated
    • Offside Tavern — one recurring event listed on their Website but not otherwise pollable

Events hosted by Secret Loft and Offside Tavern more often than not involve loud music, and in the case of some Secret Loft events, heavy use of the hallway leading to our space. The listings for these events are formatted and decorated to indicate the possibility of noise. In addition, Eventbrite's API license requires that event listings obtained from their service include a link to their event page on Eventbrite's site. As our dashboard is non-interactive, a URL shortener is used to provide a useable link as a superscript following each such listing.

Meanwhile, Offside Tavern often serves as a venue for various Meetup Groups, more often than not in some manner tech-related.

Future Ideas: Draft a Privacy Policy

Slack Calendar

The remaining calendar is generated by combining all of the above sources, and stripping away much of the formatting and decoration used in the dashboard's HTML panels. Twice a day, the Bricodash bot posts all events from both of the above calendars due to occur in the next 72 hours to the channel #thehaps on our Slack, first deleting its prior post of the same.

Also planned are Slack Events API bot functionality, such that the same calendar data can feed bah-style responses to user questions like "What's happening?"

Future Ideas: Slack Calendar Response Bot

Creepy Eyeball

The same feature that supports the dashboard door camera feed also allows for viewing Hack Manhattan's Space Cam. When viewed access the MJPG feed through the dashboard's bricolage gateway, an animated GIF of a disembodied eyeball appears as an overlay on the upper right corner of the Calendar panel.

Unlike the bo.x0.rs-hosted sousveillance system that inspired it, the Bricodash sous/veil system tracks both the connect and disconnect time of live streams. The eyeball animation appears and fades quickly for snapshots, appears and fades slowly for live streams while they are active, and then fades away more swiftly after a disconnect. Multiple simultaneous feeds have an additive effect, such that the eyeball will remain opaque longer and fade less rapidly, than for a single feed.

It should be noted that the Creepy Eyeball is only activated when our Space Cam is viewed via the Bricodash-bundled gateway. Snapshots and streams accessed via the bo.x0.rs gateway—such as link provided for viewing the Space Cam via VLC Media Player, or snapshots provided by the "Who's at the space?" trigger on our Slack—do not register as Creepy Eyeball activity.

Future Ideas

  • MJPEG Gateway Server
  • Full conversion of former sousveillance gateway references to sous/veil gateway would be ideal.

Weird Photos

At random intervals, a photo overlay will fade-zoom out from the center-left of the dashboard, remain briefly, and then fade--zoom back toward the center and disappear.

Due to limited screen real-estate, there is no ready way to share photos related to our space on the dashboard. This may change once an door buzzer IR remote solution is implemented. Until then, the Weird Photos mechanism is a compromise solution. Photos are displayed long enough to grab attention, but not so long as to obscure calendar or slack panel content for a prolonged period. Further, photos, when they do fade-zoom into view, are offset to the left to avoid obscuring the Door Cam feed.

Presently, photos are only drawn from the 100 most recently posted images on our Meetup group. (Instagram integration is pending.)

One of our members was conducting a year-long project involving representing holidays and notable events with tableaus involving rubber chickens and rubber ducks—often blue screened into appropriate settings. Many of these photos were posted to the albums for then-upcoming Open House events on the Meetup Group, hence the prevalence of rubber fowl in many of the photos that fade-zoom into and out of view on our dashboard.

Future Ideas

 

Caveats and Policies

Bricodash incorporates features from a wide variety of APIs, many of which have specific requirements for use under their respective licenses. In particular, some of the APIs used by Bricodash require that any public Web site that integrates with each such API must have a published privacy policy congruent with that governing such API's Web service. At present, no such privacy policy has been created for Bricodash.

Therefore, to remain in compliance with those licenses, Bricodash is only to be used for the on-site displays within Hack Manhattan, and not displayed on any public Web site. (At least until such time as an appropriate privacy policy has been published for Bricodash).

Future Ideas: Draft a Privacy Policy