By Gabe Weiss, Developer Advocate
One of the best parts about working at Google is the incredible diversity of interests. By day, I’m a Developer Advocate focused on IoT paid to write code to show you how easy it is to develop solutions with Google technology. By night, I’m an amateur musician. This is the true story of how I combined those two interests.
It turns out I’m not unique; a lot of folks here at Google play music on top of their day job. Before I started working here, a Googler decided that we needed a place to jam, and thus was born one of our legendary Google perks: Sound City, a soundproof room in the middle of our San Francisco office. It’s an incredible space to jam, with one catch. You can’t book the room during the day. Anyone can go in and play at any time.
This was done for two reasons: to give everyone an opportunity and to foster the magic that sometimes happens when a random set of musicians ends up in the room at the same time, resulting in a jam of epic proportions.
Some of us, however, are not yet the musical gods we aspire to become. I picked up accordion recently. Finding time to practice at home is tough as it’s not the kind of instrument you can practice after the kids go to sleep. Having access to a soundproof music room in which to practice is awesome, but I don’t necessarily want to play when other people are in the room. I don’t want to subject anyone else to my learning of the accordion. That would just be cruel.
I brought this up to the folks that run the room, and suggested putting in a camera so we can see what’s going on in the room. They said that this came up usually once a year. Folks pushed back, because no one wanted to be watched while in the room. Sound detection in the room has the same problem. If something can pick up sound, it could theoretically record sound so folks vetoed it.
Being one of the IoT device folks at Google, I asked if anyone had considered just setting up motion sensors, and having it update a page that folks could look at. A lot of staring and blinking later, my idea was passed around to the folks that run Sound City and I got a thumbs up to go for it. So now I just needed to create a motion sensor, and build a quickie webpage that monitors the state of motion in the room for folks to access.
The setup
This little project allowed me to tick all kinds of fun boxes. First, I 3d printed the case for the
Raspberry Pi +
Passive Infrared sensor (PIR). I took an existing model from Tinkercad, and modified it to fit my design. Using their tools I cut out an extra circle to expose the PIR, and
extruded the whole case model. The
end result worked great. I had plenty of room for the Pi + PIR (and added an LED to show when motion is detected to make it easier for me to debug).
The next step was to set up my data destination. For ease of setup and use, I decided to build this on
Firebase, which makes it super easy to set up a real-time database. Like all things popular and open, there are several Python libraries for connecting and interacting with Firebase, including the Google supported
Firebase Admin SDK. From a previous project, I had experience using a library called
pyrebase, so I stuck with what I knew.
Creating a Firebase project is easy: from the
console literally just add a project, give it a name and
POOF, you’re done. Note that if you already have a GCP project, you can totally just use that. Just select the project from the dropdown in the Add Project dialog instead. I went ahead and created a project for the music room.
There’s a lot of fun stuff in here to play around with if you haven’t looked at Firebase before. For my purposes, I wanted to use the real-time database. First things first, I grabbed the JSON web config to work with my project. Clicking the “Add Firebase to your web app” button from the project overview home page gave me a hunk of code like so:
<script src="https://www.gstatic.com/firebasejs/4.10.0/firebase.js"></script>
<script>
// Initialize Firebase
var config = {
apiKey: "<super sekret API key>",
authDomain: "my-project-2e534.firebaseapp.com",
databaseURL: "https://my-project-2e534.firebaseio.com",
projectId: "my-project-2e534",
storageBucket: "my-project-2e534.appspot.com",
messagingSenderId: "<numerical sender id>"
};
firebase.initializeApp(config);
</script>
I copied that to the clipboard; I will need it later.
Next up, I needed to protect the database properly. By default, the read/write rules require a user to be authenticated to interact with the database. This was perfect for ensuring that only my device could write to the database, but I also wanted to read the state from my webpage without having to authenticate with the database. I clicked the Database on the left to get started and then clicked the Rules tab to see the default rules for the database:
{
"rules": {
".read": "auth != null",
".write": "auth != null"
}
}
To allow anyone to read from my database (important safety tip, do NOT do this if the data is at all sensitive), I changed the rules to the following:
{
"rules": {
".read": true,
".write": "auth != null"
}
}
Firebase supports a bunch of rules you can apply to database permissions; click
here if you’re curious.
Next up, authentication! There are a few different ways I could have managed this. Firebase allows for email/password authentication, but if I did it that way, then everything is tied to an email, and that becomes unwieldy from a user-management perspective if someone else needs to administer things, or make changes.
Another approach is to use GCP service accounts, which Firebase honors. Now, I won’t lie, service accounts are not the easiest things to wrap your head around as there are a LOT of knobs you can turn permissions-wise in GCP/Firebase.
Here’s a good primer on service accounts if you’re interested in knowing more. I may need to write or find a good blog about service accounts at some point too. If you do go down this route, when you create the service account, be sure to check the box that says “Furnish a new private key”. There may be a message saying “You don’t have permission to furnish a private key.” Ignore that. Just be sure when you’re creating the service account, that you don’t give it full owner privileges on the entire project. You want to limit its access. For mine, I just set “Project->Editor” permissions. Even this is probably too wide open for most production uses. For this project, which is limited in scope and isolated network-wise, I wasn’t too concerned.
Once I created my service account and got my private key (JSON format), I copied and moved the key onto my Pi. So now auth with Firebase was (in theory) all set from a file standpoint.
The code
Next up, code! As a reminder, there were two things I wanted out of the data:
- To know if there was anyone in the room
- To view occupancy over time
So, funny story. PIR sensors are notoriously finicky. Like, for funsies, do a search for “PIR false positive”. You’ll find a TON of references to people trying to solve the problem of PIR sensors inexplicably triggering even when encased in a freakin lead box. In my case, the motion triggering came like clockwork. Almost every minute (+/- a couple of seconds) came a spike of motion. After an incredible amount of debugging and troubleshooting, it SEEMED like it might be power related, as I could fool it into not happening with some creative wiring, but no combination pull-up, pull-down, capacitor, resistor fixed the problem permanently. Realizing the absurdity of continuing to bang my head against a hardware problem, I just solved it in software with some code around ignoring regular pings of motion. But I’m getting ahead of myself.
Here’s the hunk o’ pertinent code that does the work from the device. There’s nothing super fancy in here, but I’ll walk through a few of the pieces:
while True:
i = GPIO.input(gpio_pir_in)
motion = 0
if i == 1:
motion = 1
# Need a chunk of code to account for a weirdness of the PIR
# sensor. No matter what I've tried, I'm getting a blip of motion
# every minute like clockwork. The internet claims everything from
# jumper position (H v. L), power fluctations, etc. Nothing offered
# seems to work, so I'm falling back on a software solution to
# discount the minute blip
current_time = int(round(time.time()))
formatted_time = datetime.fromtimestamp(current_time).ctime()
if motion:
print ("I have motion")
print (" My repeat time is: {}".format(datetime.fromtimestamp(repeat_time).ctime()))
if repeat_time == 0:
repeat_time = current_time
print(" First time for repeat: {}\n".format(formatted_time))
elif current_time >= repeat_time + 55 and current_time <= repeat_time + 65:
print (" Repeat time: {}\n".format(formatted_time))
needs_updating = 1
time.sleep(1.0)
continue
else:
print (" Real motion: {}\n".format(formatted_time))
repeat_time = current_time
elif needs_updating:
needs_updating = 0
repeat_time += 60
else:
if current_time > repeat_time + 90:
print ("No motion, but updating repeat time\nUpdating to: {}\n".format(datetime.fromtimestamp(repeat_time + 60).ctime()))
repeat_time += 60
The core of the script is a while loop that fires once a second. I wrote this bit of code to ignore the regular false positives that happen every minute, give or take. It’s intentionally not quite perfect, in that if it detects new motion that isn’t part of the cycle, it resets the cycle. This also means that motion that happens to occur a minute apart might be falsely ignored. The absolute accuracy that I sacrificed for simpler code was a fine compromise for me. If there’s one thing I can recommend, it’s to make life easier for future-you. An algorithm, or some logic, doesn’t have to be perfect if it doesn’t have to be. If it makes your life easier later, then that’s totally fine. Don’t let anyone tell you differently.
In this case, the potential faults could include:
- Real motion that occurs at one minute of time but is interpreted as false motion; this would likely self-correct within the next minute (the odds of it happening over and over are astronomically small).
- A false positive motion when real motion occurs and resets our one-minute false motion timer. Then, less than a minute later, an actual false motion happens, which is interpreted as a real motion (because it's been less than a minute since the previously detected motion). In this case, either there is consistent motion happening due to someone being in the room, or someone has just left the room and real motion has stopped, but the false motion was triggered. In the latter case, this just means less than a minute of extra detected motion before the timed pattern kicks back in and is ignored.
In other words, neither of these faults are a big deal.
One aspect of this code that took me some time to debug (even when I THOUGHT I had fixed it) is the 'else' statement at the end. The PIR doesn’t always fire every minute. There were some cases when the PIR would go past one minute without firing, which would cause my test on time at the next minute (and all subsequent minutes) to fail.
# Turn on/off the LED based on motion
GPIO.output(40, motion)
# If the current motion is the same as the previous motion,
# then don't send anything to firebase. We only track changes.
if current_motion == motion:
time.sleep(1.0)
continue
previous_motion = current_motion
current_motion = motion
try:
firebase = pyrebase.initialize_app(config)
db = firebase.database()
if motion == 1:
db.child("latest_motion").set('{{"ts": {} }}'.format(current_time))
db.child(firebase_column).push('{{"ts": {}, "device_id": {}}, "motion": {} }}'.format(current_time, device_id, motion))
except:
e = sys.exc_info()[0]
print ("An error occurred: {}".format(e))
current_motion = previous_motion
time.sleep(1.0)
Here is the second half of the while loop. It uses the config blob I saved earlier with a couple changes:
config = {
apiKey: "<super sekret API key>",
authDomain: "my-project-2e534.firebaseapp.com",
databaseURL: "https://my-project-2e534.firebaseio.com",
storageBucket: "my-project-2e534.appspot.com",
serviceAccount: “<local path to service account json>”
}
The service account handles the project ID and the sender ID pieces, so they aren’t needed in the config.
The rest, is nice ‘n’ simple. If the current motion detected is the same as the previous, don’t do anything else (I only cared about changes in motion as markers). I wrapped the Firebase connection and publish code in a broad try/catch because both can raise exceptions. But much as I didn’t care about perfect accuracy in the PIR correction code, the same goes for these exceptions. If an exception is thrown, it means one particular data point didn’t make it to the server, but this is fine because the code resets current_motion in the exception handling so that it will just try again in a second. So again, a couple seconds of being “wrong” is just fine in favor of simpler code.
The visualization
Web hosting (for the page that shows if someone is actually in the room) is SUUUPER simple on Firebase. Click the “hosting” tab on the left, and click the “Get Started” button. It leads you by the hand through installing firebase-tools from the command line, allowing you to run all of Firebase’s magical commands. First, there’s
firebase login to auth from CLI, then
firebase init to put the framework in the current directory. It’s a firebase.json file, and a www directory. If you type
firebase serve it starts up a local server that you can use to test out your page as you work. Don’t forget, there’s fairly intense caching that can happen, although it seemed to feel inconsistent to me. If you don’t see changes being made, just kill the process and restart the server with
firebase serve.
Even though this post is already pretty long, I wanted to at least talk through how to build a webpage which listens to Firebase data changes:
<!-- update the version number as needed -->
<script defer src="/__/firebase/4.5.0/firebase-app.js"></script>
<!-- include only the Firebase features as you need -->
<script defer src="/__/firebase/4.5.0/firebase-auth.js"></script>
<script defer src="/__/firebase/4.5.0/firebase-database.js"></script>
<script defer src="/__/firebase/4.5.0/firebase-messaging.js"></script>
<script defer src="/__/firebase/4.5.0/firebase-storage.js"></script>
<!-- initialize the SDK after all desired features are loaded -->
<script defer src="/__/firebase/init.js"></script>
<script src="https://www.gstatic.com/firebasejs/4.5.0/firebase.js"></script>
<script>
// Initialize Firebase
var config = {
apiKey: "<super sekret API key>",
authDomain: "my-project-2e534.firebaseapp.com",
databaseURL: "https://my-project-2e534.firebaseio.com",
storageBucket: "my-project-2e534.appspot.com",
messagingSenderId: "<ID_NUM>"
};
firebase.initializeApp(config);
This is the Node.js configuration script to initialize the Firebase object. If I hadn’t set the authorization for
read: true before, I’d need to go through authorization here as well.
Now that we’re all initialized, there are some events that we can listen to to get things rolling:
var occupied = firebase.database().ref('latest_motion');
occupied.on("value", function(data){
The document I’m updating in Firebase is that
latest_motion piece. Whenever the
latest_motion value changes (in my case, the timestamp of last motion detected from the device), that function gets called with the JSON output of the document.
Now, I didn’t HAVE to do that. I could have just made it a fully static page and required folks to hit the refresh button instead, but that didn’t seem quite right. Besides, as an IoT person, I don’t get a lot of opportunity to play with web front-ends.
If I were building a production system, there would definitely be some changes I’d need to make to this. But I just want to know if I can go practice my accordion without interrupting someone else who’s already jamming. Someday, I’ll be good enough that I’ll go up when there is someone jamming so we can jam together.
You can find all the device code and the Firebase web page project I used in my GitHub repo
here. I also talk IoT and life on my Twitter:
@GabeWeiss_.
