Data Collection


connection diagram

The Commons is outfitted with multiple smart building systems that allow us to view and, in some cases, track data over time. To make informed decisions about how to manage the building, we need to be able to look at and analyze any data in the past (how else will we know if energy usage is improving?). This website is our solution. It consolidates all of our data into a single format, a single interface, and with indefinitely long storage. There are, of course, existing tools for data management, but building our own site is much more "hands-on!"

Common Format

As indicated in the diagram above, the two main problems of dealing with the systems' interfaces directly are that 1) they have no storage or temporary storage, and 2) they all use a different interface. To solve these, we keep our own database, and run "scrapers" that grab-and-store new data from each system about every 30 seconds. After 24 hours, the data are averaged into 20-minute intervals to keep the total volume down.

Each subsystem has its own "table" in the database. Think of a table like a single spreadsheet with categories in the columns and each row contains a single data point. Some subsystems have further divisions to what we call "indexes." For example, the VRF subsystem has an index for the rooms of the building. Every 30 seconds, a new data point is fetched and stored for each room.

Custom sensors

Initially, the commons was outfitted with smart HVAC, Electrical monitoring, and solar power monitors. But these are not all we are — or want to — monitor! Part of the ongoing learning process that is running the building is realizing that we should be monitoring something that we aren't yet. To this end, we have the ability to make and install new sensors anywhere we have an internet connection! The Greenhouse DHT subsystem, for example uses an Arduino to talk to sensors throughout the greenhouse, and makes its data available to the server's scrapers. The code is accessible to about the middle-school level using our Arudino NetworkSensor Library.

We hope to see new sensors installed to monitor the geothermal system and water usage.

About this website


This website is what we would call a "data-driven" website. This means that all of the content on the data-related pages is generated on-the-fly by looking things up in the database. The website was made using a web framework called Django. Whenever anyone visits a page, it causes a piece of code to be executed on the server, often returning custom content back to the user! The source code for this website is all open-source and available online on Github.