Although Glitch is a really powerful platform where you can build full-stack web apps, I also find that one of its simplest, most straightforward features has some pretty amazing potential.

You can easily create and maintain a group-edited collection of Markdown files on Glitch, almost like a Google Docs for Markdown. Once you do, there's a huge set of possibilities of what you can do with those files

If you're here, you already know what Markdown is — it's a way of formatting plain-text files so that they can consistently be parsed into rich text or HTML if you'd like. I've been using Markdown since John Gruber first created it almost 2 decades ago, and it's been amazing to see it become so ubiquitous that many more technical users just assume it's going to be available wherever they want to write.

Markdown also has a few key advantages over the ways that people commonly write together as part of teams or workgroups. In a tool like Google Docs, as good as it is, there are some pretty significant constraints:

There's no built-in formatting styles for code, or even for blockquotes.

You can't just use HTML even if you know it

Copy & paste can still be annoying in the same way that Microsoft Word is frustrating, bringing over unexpected styles or formatting

None of these by itself is a huge problem, but all of them together can be really annoying and frustrating. Meanwhile, trying to edit Markdown files alongside other people is... weirdly hard? You could use a whole complex git-based workflow, but that seems like overkill, and we're spoiled by being used to edit together in Google Docs and Figma and all our other tools that let us collaborate in realtime.

So, the answer to easily editing Markdown files while working together on them as easily as sharing a Google Doc? Glitch!

Take the piece that you're reading right now: I wrote it right on Glitch, and then I could easily share it with my coworkers so they could read over it and offer edits or suggestions if they wanted. It works just like sharing in any other app, without having to go through the complexity of git. And now if I don’t want my notes to be readable by everybody on the Internet, I can just set sharing to be private, and only the people I invited can read what I’ve written.

Now, if I want to undo changes, or go back to an earlier version, I still have git under the hood, automatically saving all my edits as I go. If I want to get really fancy, I can even hook up other tools to my Markdown files on Glitch because it's all accessible as a completely standard git repository. But I don't have to do any of that stuff unless I want to.

And best of all, any people I invite in to share this Glitch project can also create and share their own Markdown files that we all maintain together. If we want to make a knowledge base or a documentation manual together, it's easy enough for a brand new user to jump in, and advanced enough for any coder to feel fully comfortable building on top of. There are even niceties like a fancy little button that will automatically format your file and keep it looking neat and clean.

Of course, since Glitch lets you create and run full-stack apps, the sky's the limit when it comes to using the Markdown files you make on Glitch. If you want, you can build (or import from GitHub) a static site generator, and publish your Markdown files as web pages. You could use the built-in git support to synchronize the repo to another hosting platform or publishing tool. You could write scripts to process your Markdown and output it in any format you want, or to submit it to another platform's APIs. You could even build another Glitch app that uses your documents as data that it works from.

The key thing is, it's just code. You can do anything any standard coding environment lets you do, but it's just easier to get started writing and creating and previewing your Markdown files.

So, if you're looking for an easy way to create and share documents together, and Google Docs isn't a good fit because it's not designed for plain text files, keep Glitch in mind as part of your toolkit. (And let us know if you find good uses for it with your team!)

Although Glitch is a really powerful platform where you can build full-stack web apps, I also find that one of its simplest, most straightforward features has some pretty amazing potential.

You can easily create and maintain a group-edited collection of Markdown files on Glitch, almost like a Google Docs for Markdown. Once you do, there's a huge set of possibilities of what you can do with those files

If you're here, you already know what Markdown is — it's a way of formatting plain-text files so that they can consistently be parsed into rich text or HTML if you'd like. I've been using Markdown since John Gruber first created it almost 2 decades ago, and it's been amazing to see it become so ubiquitous that many more technical users just assume it's going to be available wherever they want to write.

A Google Docs for Markdown

Starting today, Glitch members with paid subscriptions can easily control who accesses their code and who sees live projects. 

We’ve made sharing your app easier than ever. Control who has access and who can edit a project all without having to build some complicated authentication system for your project.

Here are just a handful of things you can do with this feature:

Stay in stealth mode until launch. Developer advocates can build demos and keep them private until the big announcement. 

Share code with students or workshop attendees. From classrooms to conferences, you can control who sees assignments and projects.

Build internal tools. Whether you’re prototyping a new feature or using Glitch for your latest greatest Slack integration, create apps with your colleagues in mind.

Interview candidates. We’ve heard from teams who use Glitch to evaluate new team members. Now it’s even easier to share code examples with potential hires.

Grow your own private blog or newsletter. Have something you want to save for your inner circle? Share your best ideas with a select audience.

Build a project and hit the “Share” button in the top left just like you normally would. You’ll see a new pop up with a drop down menu where you can set your project’s sharing settings. 

Public is the default setting, which makes your project visible and its code remixable for everyone 

Private limits both your code and live site to project members

Private Code keeps the live site visible, but restricts code to project members 

We’ve eliminated the ability to join projects with share links and instead, are now asking project owners to explicitly invite collaborators via email. This implementation is more secure and gives projects owners greater visibility into who has access. 

Current free users with “private” projects will still have their code only visible to members of their project. However, changing a project from private to public on a free account can’t be reversed. 

We’re spending a lot of time thinking about how to better enable groups of users on Glitch and what collaboration looks like in a professional setting. Since we launched our paid tier in the spring, we’ve seen lots of individual creators build more powerful apps and also introduce Glitch into some of their professional workflows - from hiring and onboarding team members to prototyping new features and internal tools. We’ll share more on this front as our roadmap evolves.

If you have any fun or interesting projects you’re building using today’s news, feel free to tag us on Twitter or email community@glitch so we can share with the community! 

Starting today, Glitch members with paid subscriptions can easily control who accesses their code and who sees live projects.

The easiest way ever to control who you share your app with

2020 is winding down, but industry events are far from over. And unlike previous years where attending a conference might require getting on a plane or spending all of that professional development budget, online events make attending easier than ever. 

If you’re a developer - especially one interested in javascript, open source, or general web dev - here are some upcoming dates and happenings to put on your radar. 

Advent of Code - December 1-25, Asynchronous

Already under way, Advent of Code, is a series of daily coding challenges that makers of all skill levels can participate in and use any programming language to solve. Similar to #100daysofcode, you can use advent of code to test your speed, sharpen new skills, or participate in a little friendly community competition. The first 100 entries each day get added to a leaderboard. 

Github Universe - December 8-10, 9 - 1:30 PST

With four distinct tracks, you’re sure to find something that piques your interest at this year’s virtual event. The developer track is tailored to open source contributors and maintainers, the enterprise track is for senior leaders and decision-makers, the play track is for creative coders, and last but not least, the university track is for students and teachers. 

If you’re interested in general trends related to open source and the broader developer landscape, add the keynote to your calendar (12/8, 12 PM EST and check out this year’s State of the Octoverse report. 

Learn about updates coming to Chrome and watch speakers share protips on the latest tools and best practices for web devs. We’re looking forward to seeing some really well-produced videos; Googlers working on Chrome have impressed us in years past with their talk, both virtual and in-person. Request a free invite and keep your eyes peeled for some Glitch apps that might make an appearance!

You Got this (from your couch) - January 15 - 16

Kick off the new year with some fresh takes on how to find work-life bliss. The You Got This conference series is dedicated to helping you develop "core skills needed for a happy, healthy work life." The upcoming conference in January is split into paid workshops on Friday (£40 per workshop or £90 for the full day) and free sessions on Saturday, the 16th. The topics are all related to 7 overarching themes: professional development, self care, working with others, ethics in tech, self advocacy, how to best support your teams as managers and diversity, and accessibility & inclusion. Free tickets available here. 

Node Congress - February 18 - 19 (Call for speakers open!)

If you want to spend two days focused on all-things Node, this event is for you. The full schedule hasn’t been published yet, but you can reserve a free ticket today. And if you have something you want to share with the community, you have until January 10 to submit a speaker proposal. 

According to the official website, MozFest is “a unique hybrid: part art, tech and society convening, part maker festival, and the premiere gathering for activists in diverse global movements fighting for a more humane digital world.” Organizers recently made the call to shift from Amsterdam to a virtual event this spring. We attended last year and loved being surrounded by thousands of developers who were just as passionate about making a better, healthier Internet as we are. Call for speakers is officially closed and the schedule should be posted in the beginning of January.

2020 is winding down, but industry events are far from over. And unlike previous years where attending a conference might require getting on a plane or spending all of that professional development budget, online events make attending easier than ever. 

If you’re a developer - especially one interested in javascript, open source, or general web dev - here are some upcoming dates and happenings to put on your radar.

Upcoming Developer Events and Happenings

We’re taking a break from our weekly round up of noteworthy apps to share a post written by Mads Hartmann, the Engineering Manager for our platform team. From time to time, we’ll post articles written by Glitch employees that give you a sneak peek into some of the behind-the-scenes work that goes into making Glitch go and grow. 

Over the past year, Mads has written a multi-part series about observability at Glitch and the tools and processes we’ve implemented to make the site more reliable. The most recent post goes into a deep dive on SLO-based alerts - challenges with our previous on-caller system and an excerpt of our internal tech spec. There’s also a getting started guide that’s a good primer if you’re not familiar with SLOs or want to learn more about what goes into their implementation.

“We’re hoping that by implementing SLOs - and alerting on them - we’ll be able to improve communication during incidents, reduce the toil on on-callers, and help improve our reliability in a way that’s meaningful to our users.”

Read the full post here. And if you want to work alongside talented and thoughtful engineers like Mads, we’re hiring. 

Adopting SLO-based alerting on Glitch

Throughout the election season, we saw data from a small number of states captivate the attention of a global audience. Social media feeds were a constant stream of numbers and maps; it was nearly impossible to make sense of it all in real-time. Instead of getting overwhelmed, Daniel Flanagan created Too Close to Call, a Glitch project that does all the marginal math needed to follow close races in competitive states. I reached out to Daniel, a recent college graduate and freelance visual effects artist, to talk about his process and the aftermath of creating a popular app during, and for, such an energizing national moment.

Jenn: With every election there are lots of maps and data–I, for one, often get overwhelmed. What drove you to create your own app among all that noise?

Daniel: That feeling of data overload is exactly what drove me to make this app.  I knew the election was going to come down to a handful of states, but once early counts confirmed as much, I, and I’m quoting myself here, “was annoyed with constantly searching” a whole map of the country when only a few states were relevant.

Add to that the fact that all these maps would only tell you which way the state was leaning and maybe a percentage/vote count if you selected a specific state, it was a pain to find the information that was truly relevant.  I wanted to know how close each state’s race was: 200 votes or 200,000 votes?

I do appreciate the work that goes into all those maps, I follow a few of their developers on Twitter and I think they knocked it out of the park.  My little app is not possible without all their work and the ease with which those maps convey the state of the race at a glance is incredibly valuable.  In this case I just wanted a more concise version of only the data I found relevant.

You mentioned in the support forum that you hold a degree in political science! Do you often build apps that are politics related, or was this a special opportunity for you?

I did yes. Technically it's a minor in political science but I definitely got more out of my poli sci classes than any others.  To be honest, I don’t build all that many apps in the first place, this sort of user-facing development is a hobby more than anything else.

I build things around stuff that interests me, and to say that I was interested in this election would be an understatement.  The challenging thing with anything political is the fact that everyone comes to it with their own politics.  I’m not looking to make something that drives people apart, or even something that affirms someone’s beliefs; the interesting projects to me are the ones that provide the opportunity to bring understanding and context to an area, as Malcolm Gladwell would say, otherwise overlooked or misunderstood.

This was a special opportunity for numerous reasons, chiefly because it allowed me to channel my election energy into something productive.  I was also excited to be working with real-time live data, that’s not something I get to do often.

How long was the process from getting the idea to sharing it with the world?

I had the idea as I was scrolling through Twitter that morning and set up the project on Glitch shortly thereafter.  I didn’t get to start in earnest until lunch around noon, and my work on this project was sporadic throughout the day as I was working on, well, work.  I’d say it probably got 4-5 hours of dedicated attention throughout the day. I took a look at my commit history and it looks like the first commit was at 10am, the UI was done by about 2pm, I made a commit noted “Functional Prototype” just before 5pm, and I shared it on Twitter an hour later at 6:03pm. So from commit to tweet we’re looking at 8 hours and 3 minutes. 

What made Glitch the place you chose to build and share this project?

For any project or idea that requires a web server, Glitch is always my first choice.  I don’t quite understand how Glitch is able to provide so much for free but I’m not complaining.  Glitch containers allow for websocket connections, terminal access, and persistent storage, all while abstracting the infrastructure away from me. Aside from the technical, I feel like my kind of app development, small, one-off experiences fits so well within the Glitch community.  I’ve come across so many awesome small projects on Glitch and being a part of that is something that’s hard to find on the internet.

Did you know how you were going to build the app when you got started or did those decisions come to you as you were coding? For instance, did you go into the process knowing you’d be using WSJ data?

I didn’t start this app until I had the Wall Street Journal data source, which I didn’t even find myself, it came across my Twitter feed courtesy of Paul Miller. I knew when I started this was going to be a Flask based Python app because I’m super familiar with that setup and how it interacts with Glitch.  I built out the UI first, and comparing what it looks like now to a sketch in my notebook, I can confidently say that I built exactly what I set out to build.

Updating the data went through a few design ideas:  First I thought I’d have the data be filled out server-side and sent to the client using Flask’s built-in Jinja templates.  The biggest benefit to that approach was not having to write JavaScript.  JavaScript is a great language and it powers a lot of what I do, but I really don’t like writing it.

Eventually I came to my senses and decided the flexibility of being able to update the data without a full page reload was worthwhile.  I did that with a Flask endpoint and a fairly simple client JavaScript file.  Ironically I built in this flexibility and didn’t take advantage of it until I started getting some feedback on the app.

Once you shared the app, it really took off! How was it received, and did you need to make changes to the app in response to the reception? Was there anything you would have done differently given more planning and time?

It was so exciting, and a little frightening, to see this app take off.  It was a crash course in social media virality for me; I tweeted it to my 81 followers and to date it’s reached 3,000 unique users totaling more than 7,000 page views.

The reception was awesome, it was great to see so many people using it, and it turned out a lot of people were looking for something to cut out all the noise for them.  One of my best friends sent me a photo of the app on his company’s office flat screen and told me how the app flickered as they refreshed it.  Conveniently, I had built in the flexibility to have the app update the data automatically, I just hadn’t implemented it.  That’s where the two minute refresh countdown came from.

In response to comments on Twitter I added the electoral vote counts, the map color changes, and the minimize maps toggle.  I also added PWA (Progressive Web App) capabilities after I kept losing the tab on my phone.

I enjoyed the time pressure, knowing that at any moment it could become irrelevant was a motivator to get it done.  The one thing I wish I did was cache the data and show some sort of trend line from it.  Even something as simple as showing which direction the vote margin was trending would have been awesome.  There’s a similar app another group of developers made that was brought to my attention by people commenting on my app on Twitter that I think does that really well.

This week I’ve been a part of an awesome Glitch ✕ Twitter collaboration on an in-browser Jeopardy board.  I adapted an old API of mine that allows a frontend developed by Patrick Weaver and Trent Willis to pull clues from previously aired episodes of Jeopardy.  You can check out my API, the gameboard, and the Twitter thread.

Other than that I’ve got a lot of ideas but no concrete plans as of yet.  Maybe something similar to this app for the upcoming Georgia Senate runoff elections. Although answering these questions for the Glitch blog is making me think that maybe I want to have my own blog 👀.

Thanks to Daniel, who should most definitely start his own blog, for telling his story and for building Too Close to Call on Glitch! If you build or see an app on Glitch that you want us to feature, tell us about it by emailing community@glitch.com.

Throughout the election season, we saw data from a small number of states captivate the attention of a global audience. Social media feeds were a constant stream of numbers and maps; it was nearly impossible to make sense of it all in real-time. Instead of getting overwhelmed, Daniel Flanagan created Too Close to Call, a Glitch project that does all the marginal math needed to follow close races. 

I reached out to Daniel, a recent college graduate and freelance visual effects artist, to talk about his process and the aftermath of creating a popular app during and for such an energizing national moment.

Too Close to Call: A conversation with creator Daniel Flanagan

Last week we showed how our community has been using WebXR to create new worlds and tell stories in 3D. There’s a lot to be said, though, about how much you can create with only two dimensions. The Glitch community has not only been showing what can be done with a little bit of JavaScript and map tiles, but they’re changing the way we think about maps entirely! These projects don’t just help us get to a place, but they tell compelling stories as well.

Let’s take a look at some of our recent favorites.

Carta Marina, a map of fantastical Nordic sea creatures

Creator @mapsmania has been making some of the coolest interactive maps ever! Some are location-based like their interactive history of the London tube, but some like ~carta-marina guide us through art and history in a really unique way. Every map enthusiast would enjoy their blog, Maps Mania.

Waking and Sleeping, a world map of wakedness

This real-time map of users tweeting “good morning” and “good night” across the world does what most maps don’t do: remind us that we’re not alone. Its creator, creative technologist @bookwormgirl910, describes the project as “a reflection on world interconnectedness and our daily waves and flares of activity and wakedness across the globe.” I recommend you check out the about page of the project to learn how it works and what limitations Katherine faced around languages and geotagging with the Twitter API in particular.

Supertall Vacancies, a story of NYC housing in the shadows of luxury vacancies

This interactive map by @apr2136 and collaborators at Columbia’s Points Unknown takes a look at both commercial and residential real estate in Manhattan and vacancy trends from recent years, and even gives a literal voice to doormen who have witnessed the impact of luxury housing surplus firsthand. 

If you have a story to tell, it’s clear from the above projects that using maps is a great way to get your readers interested. And with Glitch, you can have something running in seconds–you can remix all of the above awesome map projects and more in our collection of storytelling maps. The collection also includes some apps that can get you started with the JavaScript libraries that these creators used for their own projects:

Leaflet.js, a lightweight and open source library for creating apps that are mobile friendly. We created this Guide to Leaflet including remixable apps to get you up and running.

Mapbox, a location data platform that lets you build powerful location-based apps on the web. They’re on Glitch at @mapbox and have posted some great apps to get you started!

We can’t wait to see what you create and what stories you tell–we’d love for you to let us know about them! If you have Glitch projects you’d like us to feature, tweet them to us @glitch or email community@glitch.com! ✨

The project featured in this post's header image is ~lhh by @darzhz

Worlds to Explore: Storytelling with Maps

As many of us are spending more time at home and looking at the same screen before, during and after work, a change of scenery is often welcome. Lucky for us, the Glitch community has been producing incredibly compelling immersive web experiences that take us to other worlds, tell stories, and inspire and teach us to create our own. 

Here are some of the virtual spaces I’ve found myself getting lost in over the past couple of weeks. Happy exploring! 

Groad Tour, a glimpse of Portland then and now 

Creator Nathan Backous takes us all on a virtual tour of all-ages venues around Portland, Oregon where the band JonnyX and the Groadies played at and have since closed. "Some of the venues captured in this project are separated over 20 years from their previous purpose. Some of the buildings remain relatively unchanged, some are drastically different, some are entirely gone,” writes Backous. Each “stop” contains flyers, signs, and photos from the band giving viewers a true trip down memory lane. 

The Homer Metaverse, for Simpsons fans everywhere

Something that creator Fabio Cortés and I have in common is that our first exposure to VR was an episode of The Simpsons from 1995 where Homer finds himself in a mysterious 3D world behind a book shelf. This Glitch app is a playful throwback to the series.

Jurassic Warp, a multi-media exhibit (Note: it auto-plays sound on page load)

This is an incredible A-frame art piece created by artists KT Duffy and María Luisa Sanín Peña. It’s just one of many examples of artists converting their work in response to recent gallery closings. According to New City Art, Duffy’s original exhibition was planned for Ground Level Platform in Chicago, but then the project was moved online to Glitch!

These are just a handful of projects to check out; you can find them and more in our Immersive Web Experiences collection. If you’re interested creating your own immersive web experiences, there are a lot of great resources to get you started. Here are just a few:

Check out our WebVR starter kit, which includes short videos and app examples that introduce you to WebVR and guides you through creating your own virtual solar system in A-frame.

For a guided introduction to A-frame, check out their free interactive course, A-Frame School, and you can remix any of their starter apps here on Glitch.

I’m a big fan of the Immersive Web Weekly newsletter, published by the W3C’s Immersive Web working group. Subscribers get a weekly email curated with news, events, art and projects related to the immersive web, and it’s a great way to learn more and be inspired by this community!

We can’t wait to see what you create, and we can’t wait to help you show your projects off! If you have Glitch projects you’d like us to feature, tweet them to us @glitch or email community@glitch.com! ✨

The project featured in this post's header image is ~earthy-course by @adarosecanon.

﻿﻿As many of us are spending more time at home and looking at the same screen before, during and after work, a change of scenery is often welcome. Lucky for us, the Glitch community has been producing incredibly compelling immersive web experiences that take us to other worlds, tell stories, and inspire and teach us to create our own.

Worlds to Explore: Immersive Web Experiences

Over the summer we announced fast and free static sites on Glitch, providing you a way to create and share simple webpages that never go to sleep. Static sites are great for ambitious client-side apps, landing pages for educational demos, compelling artworks from code - basically any project that needs only HTML, CSS and client-side JavaScript to come to life! Some of you have reached out asking how to turn your full-stack JavaScript apps into static sites so you can save on Project Hours and keep your projects awake for free, so I’m going to show you how to get started. 

Glitch distinguishes between a static and full-stack site by watching for any of three simple configuration files:  package.json,  requirements.txt and glitch.json. Those two last files are for configuring apps built in other experimental-to-Glitch languages; for this tutorial I’m going to focus on full-stack JavaScript apps using Node.js, which are configured in package.json. If you’re not familiar with the anatomy of a Node app, you can check out our free zine on Making Web Apps with Node!

To get started, you’ll need a full-stack app that’s running all of its important code on the client-side. For this tutorial I created ~full-stack-to-static, whose server-side code only contains routes for where the client side files are. 

The first thing I do is remove the server-side files and see what breaks and troubleshoot from there. So, I remove package.json, letting Glitch know that this should be a static site. Then I remove server.js, and .env since they will not be run without package.json.

Going to the app view, I see that something looks very wrong. Instead of my index.html content, I see a file listing like this:

This is because my index.html file is still in a“views” folder. Renaming“views/index.html” to“index.html” moves that file in the root of my project and fixes that.

I can now see index.html rendering but I now get errors when trying to load my stylesheet and JavaScript. This is because they are inside a“public” folder but my index.html references them as if they were in the root of my project - something my server-side code had handled before. I can fix this either by updating links in index.html to add the“public/” directory, or by renaming“public/style.css” and“public/script.js” to“style.css” and“script.js”. In this example, I did the latter and now my static site is looking great. Here's an embed of the final result:

Now I have a project that does basically the same thing it was doing before, but it stays awake and it doesn’t count towards my Project Hour limits. I recommend you take a look at your Project Dashboard and see if you have any full-stack projects using up Project Hours that could be refactored to static sites! And for all your projects that need to be full-stack apps, just remember as always that Glitch gives you 1000 free Project Hours each month to run your apps, and subscribers can lift those limits if you need even more.

The project featured in this post's header image is ~iris-cockroach by @potch. Have a project you want us to feature? Tweet them to us @glitch or email community@glitch.com!

Over the summer we announced fast and free static sites on Glitch, providing you a way to create and share simple webpages that never go to sleep. Static sites are great for ambitious client-side apps, landing pages for educational demos, compelling artworks from code - basically any project that needs only HTML, CSS and client-side JavaScript to come to life! Some of you have reached out asking how to turn your full-stack JavaScript apps into static sites so you can save on Project Hours and keep your projects awake for free, so I’m going to show you how to get started...

Going from Full-stack to Static

The code for this tutorial is available on GitHub. You can also check out the live demo of the app on Glitch

The first endpoints of the new Twitter API are out and waiting for you to explore and build apps with in order to understand what’s happening in the public conversation.

This tutorial will walk you through how to build your own real-time Tweet streaming app using the filtered stream endpoints and Tweet annotations to listen for Tweets based on your own topics of interest.

After building the app, you will learn about how it can be applied to some real-life examples to inspire you to get started such as.

Discovering new music videos: Imagine a dashboard that lets you see the music videos being shared across Twitter

Finding remote developer job openings: Imagine creating a remote developer job listings app with remote developer job openings being shared across Twitter

Learning about personal finance and savings: Surface public conversations about personal finance and savings happening on Twitter

To get started here’s what you will need:

You must have an approved developer account, and have activated the new developer portal experience. If you don’t have one already, you can apply for one. Access is available with active keys and tokens for a developer App that is attached to a Project created in the developer portal. 

A Bearer Token from your App in the Twitter developer portal.

Npm (This is automatically installed with Node. Make sure you have npm 5.2 or higher.)

First, install Node.js. Check out the Downloads section from Node’s website and download the source code or installer of your choice. Alternatively, if you are running on a Mac you can install the Node package using the Brew package manager

Open a terminal window and bootstrap your React app using create-react-app by using npx. 

npx create-react-app real-time-tweet-streamer

After create-react-app has finished executing, change to the newly created real-time-tweet-streamer directory and replace the scripts block in your package.json with the following script block in your package.json. These lines will provide a command shortcut to concurrently run your client and server backend code in development or production as needed.

"scripts": {
  "start": "npm run development",
  "development": "NODE_ENV=development concurrently --kill-others \"npm run client\" \"npm run server\"",
  "production": "npm run build && NODE_ENV=production npm run server",
  "client": "react-scripts start",
  "server": "node server/server.js",
  "build": "react-scripts build",
  "test": "react-scripts test",
  "eject": "react-scripts eject"
},

After updating the scripts section, your package.json should now look as follows.

Next, remove all files within the src/ subdirectory. 

Then, create a new file within the src/ subdirectory called index.js. The code for this file will be as follows.

import React from "react";
import ReactDOM from "react-dom";
import App from "./components/App";

ReactDOM.render(<App />, document.querySelector("#root"));

Connecting to the filtered stream endpoints requires you to authenticate using a bearer token from your app in the Twitter developer portal. To utilize your bearer token, you will need to have the following environment variable set. You can do so by issuing the following command in your terminal window assuming you are using bash as your shell. Replace <YOUR BEARER TOKEN HERE>, including the left and right angle brackets, with your bearer token.

export TWITTER_BEARER_TOKEN=<YOUR BEARER TOKEN HERE>

First, you will need to get started with implementing the Node server, which will be responsible for making the actual requests to the Twitter API. This Node server will serve as a proxy between your browser-based React client and the Twitter API. On your Node server, you will need to create API endpoints that connect to the filtered stream endpoints. In turn, requests from your React client will be proxied through to your local Node server. 

Before you go any further, cd to the project root directory and install the following dependencies

npm install concurrently express body-parser util request http socket.io path http-proxy-middleware request react-router-dom axios socket.io-client react-twitter-embed

Next, while still within your project root directory, create a new subdirectory called “server” and a new file within that subdirectory called “server.js”. 

This source code file will contain all of your backend logic for connecting to and receiving Tweets from the filtered stream endpoint. The contents of your server.js file will be as follows.

Filtering Tweets on the filtered stream endpoint using rules

Filtered stream has two endpoints, a streaming endpoint, to receive data and a rules endpoint that is used to create and delete rules. The filtered stream endpoints require you to define search queries called rules, in order for it to know what kind of Tweets to send to you. Rules allow you to narrow down to only the Tweets you are looking for by using a set of operators. You will see some example use cases and corresponding rules you can use later once you finish building the app.

The other filtered stream endpoint is the streaming endpoint, which uses a simple GET connection. Once a connection is established, Tweets are delivered in JSON format through a persistent HTTP streaming connection. You will only receive Tweets matching your rules while connected to the stream. 

The next step is to work on the following React components

App.js - The parent component that be will, in turn, render all other components

NavBar.js - Displays the navigation bar for navigating between the Tweet feed and managing rules

Tweet.js - Displays a Tweet on the page 

TweetFeed.js - Renders multiple Tweet components at once in a “feed” like fashion

Rule.js - Renders an individual rule on your stream

RuleList.js - Renders multiple Rule components and displays an input field to add a rule

ErrorMessage.js - Renders any status or error messages to the screen

Spinner.js - Renders a loading indicator for any pending API calls

Now you will need to get started with creating the React components. Under your /src subdirectory, create a directory called “components”. The source code files above will be stored in this new directory. First, create the parent most component of the application. This component will be responsible for rendering all other components.

import React from "react";
import { BrowserRouter, Route } from "react-router-dom";

import Navbar from "./Navbar";
import TweetFeed from "./TweetFeed";
import RuleList from "./RuleList";

class App extends React.Component {
  render() {
    return (
      <div className="ui container">
        <div className="introduction"></div>

        <h1 className="ui header">
          <div className="content">
            Real Time Tweet Streamer
            <div className="sub header">Powered by Twitter data</div>
          </div>
        </h1>

        <div className="ui container">
          <BrowserRouter>
            <Navbar />
            <Route exact path="/" component={RuleList} />
            <Route exact path="/rules" component={RuleList} />
            <Route exact path="/tweets" component={TweetFeed} />
          </BrowserRouter>
        </div>
      </div>
    );
  }
}

export default App;


Next, create the component for the navigation bar.

import React from "react";
import { NavLink } from "react-router-dom";

const Navbar = () => {
  return (
    <div className="ui two item menu">
      <NavLink to="/tweets" className="item" target="_blank">
        New Tweets
      </NavLink>
      <NavLink to="/rules" className="item" target="_blank">
        Manage Rules
      </NavLink>
    </div>
  );
};

export default Navbar;


Next, create the parent component for rendering all job listings.

import React, { useEffect, useReducer } from "react";
import Tweet from "./Tweet";
import socketIOClient from "socket.io-client";
import ErrorMessage from "./ErrorMessage";
import Spinner from "./Spinner";

const reducer = (state, action) => {
  switch (action.type) {
    case "add_tweet":
      return {
        ...state,
        tweets: [action.payload, ...state.tweets],
        error: null,
        isWaiting: false,
        errors: [],
      };
    case "show_error":
      return { ...state, error: action.payload, isWaiting: false };
    case "add_errors":
      return { ...state, errors: action.payload, isWaiting: false };
    case "update_waiting":
      return { ...state, error: null, isWaiting: true };
    default:
      return state;
  }
};

const TweetFeed = () => {
  const initialState = {
    tweets: [],
    error: {},
    isWaiting: true,
  };

  const [state, dispatch] = useReducer(reducer, initialState);
  const { tweets, error, isWaiting } = state;

  const streamTweets = () => {
    let socket;

    if (process.env.NODE_ENV === "development") {
      socket = socketIOClient("http://localhost:3001/");
    } else {
      socket = socketIOClient("/");
    }

    socket.on("connect", () => {});
    socket.on("tweet", (json) => {
      if (json.data) {
        dispatch({ type: "add_tweet", payload: json });
      }
    });
    socket.on("heartbeat", (data) => {
      dispatch({ type: "update_waiting" });
    });
    socket.on("error", (data) => {
      dispatch({ type: "show_error", payload: data });
    });
    socket.on("authError", (data) => {
      console.log("data =>", data);
      dispatch({ type: "add_errors", payload: [data] });
    });
  };

  const reconnectMessage = () => {
    const message = {
      title: "Reconnecting",
      detail: "Please wait while we reconnect to the stream.",
    };

    if (error && error.detail) {
      return (
        <div>
          <ErrorMessage key={error.title} error={error} styleType="warning" />
          <ErrorMessage
            key={message.title}
            error={message}
            styleType="success"
          />
          <Spinner />
        </div>
      );
    }
  };

  const errorMessage = () => {
    const { errors } = state;

    if (errors && errors.length > 0) {
      return errors.map((error) => (
        <ErrorMessage key={error.title} error={error} styleType="negative" />
      ));
    }
  };

  const waitingMessage = () => {
    const message = {
      title: "Still working",
      detail: "Waiting for new Tweets to be posted",
    };

    if (isWaiting) {
      return (
        <React.Fragment>
          <div>
            <ErrorMessage
              key={message.title}
              error={message}
              styleType="success"
            />
          </div>
          <Spinner />
        </React.Fragment>
      );
    }
  };

  useEffect(() => {
    streamTweets();
  }, []);

  const showTweets = () => {
    if (tweets.length > 0) {
      return (
        <React.Fragment>
          {tweets.map((tweet) => (
            <Tweet key={tweet.data.id} json={tweet} />
          ))}
        </React.Fragment>
      );
    }
  };

  return (
    <div>
      {reconnectMessage()}
      {errorMessage()}
      {waitingMessage()}
      {showTweets()}
    </div>
  );
};

export default TweetFeed;

Next, create the child component for the previous component that renders an individual Tweet. 

import React from "react";
import { TwitterTweetEmbed } from "react-twitter-embed";

const Tweet = ({ json }) => {
  const { id } = json.data;

  const options = {
    cards: "hidden",
    align: "center",
    width: "550",
    conversation: "none",
  };

  return <TwitterTweetEmbed options={options} tweetId={id} />;
};

export default Tweet;


Next, create the component responsible for rendering all the rules on our stream as well as displaying input controls for creating new rules. In this case, we will only be using one rule.

import React, { useEffect, useReducer } from "react";
import axios from "axios";
import Rule from "./Rule";
import ErrorMessage from "./ErrorMessage";
import Spinner from "./Spinner";

const reducer = (state, action) => {
  switch (action.type) {
    case "show_rules":
      return { ...state, rules: action.payload, newRule: "" };
    case "add_rule":
      return {
        ...state,
        rules: [...state.rules, ...action.payload],
        newRule: "",
        errors: [],
      };
    case "add_errors":
      return { ...state, rules: state.rules, errors: action.payload };
    case "delete_rule":
      return {
        ...state,
        rules: [...state.rules.filter((rule) => rule.id !== action.payload)],
      };
    case "rule_changed":
      return { ...state, newRule: action.payload };
    case "change_loading_status":
      return { ...state, isLoading: action.payload };
    default:
      return state;
  }
};

const RuleList = () => {
  const initialState = { rules: [], newRule: "", isLoading: false, errors: [] };
  const [state, dispatch] = useReducer(reducer, initialState);
  const exampleRule = "from:twitterdev has:links";
  const ruleMeaning = `This example rule will match Tweets posted by 
     TwtterDev containing links`;
  const operatorsURL = "https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/integrate/build-a-rule";

  const rulesURL = "/api/rules";

  const createRule = async (e) => {
    e.preventDefault();
    const payload = { add: [{ value: state.newRule }] };

    dispatch({ type: "change_loading_status", payload: true });
    try {
      const response = await axios.post(rulesURL, payload);
      if (response.data.body.errors)
        dispatch({ type: "add_errors", payload: response.data.body.errors });
      else {
        dispatch({ type: "add_rule", payload: response.data.body.data });
      }
      dispatch({ type: "change_loading_status", payload: false });
    } catch (e) {
      dispatch({
        type: "add_errors",
        payload: [{ detail: e.message }],
      });
      dispatch({ type: "change_loading_status", payload: false });
    }
  };

  const deleteRule = async (id) => {
    const payload = { delete: { ids: [id] } };
    dispatch({ type: "change_loading_status", payload: true });
    await axios.post(rulesURL, payload);
    dispatch({ type: "delete_rule", payload: id });
    dispatch({ type: "change_loading_status", payload: false });
  };

  const errors = () => {
    const { errors } = state;

    if (errors && errors.length > 0) {
      return errors.map((error) => (
        <ErrorMessage key={error.title} error={error} styleType="negative" />
      ));
    }
  };

  const rules = () => {
    const { isLoading, rules } = state;

    const message = {
      title: "No rules present",
      details: [
        `There are currently no rules on this stream. Start by adding the rule 
        below.`,
        exampleRule,
        ruleMeaning,
      ],
      type: operatorsURL,
    };

    if (!isLoading) {
      if (rules && rules.length > 0) {
        return rules.map((rule) => (
          <Rule
            key={rule.id}
            data={rule}
            onRuleDelete={(id) => deleteRule(id)}
          />
        ));
      } else {
        return (
          <ErrorMessage
            key={message.title}
            error={message}
            styleType="warning"
          />
        );
      }
    } else {
      return <Spinner />;
    }
  };

  useEffect(() => {
    (async () => {
      dispatch({ type: "change_loading_status", payload: true });

      try {
        const response = await axios.get(rulesURL);
        const { data: payload = [] } = response.data.body;
        dispatch({
          type: "show_rules",
          payload,
        });
      } catch (e) {
        dispatch({ type: "add_errors", payload: [e.response.data] });
      }

      dispatch({ type: "change_loading_status", payload: false });
    })();
  }, []);

  return (
    <div>
      <form onSubmit={(e) => createRule(e)}>
        <div className="ui fluid action input">
          <input
            type="text"
            autoFocus={true}
            value={state.newRule}
            onChange={(e) =>
              dispatch({ type: "rule_changed", payload: e.target.value })
            }
          />
          <button type="submit" className="ui primary button">
            Add Rule
          </button>
        </div>
        {errors()}
        {rules()}
      </form>
    </div>
  );
};

export default RuleList;


Next, create the child component of RuleList.js responsible for displaying a single rule and deleting a rule.

import React from "react";

export const Rule = ({ data, onRuleDelete }) => {
  return (
    <div className="ui segment">
      <p>{data.value}</p>
      <div className="ui label">tag: {data.tag}</div>
      <button
        className="ui right floated negative button"
        onClick={() => onRuleDelete(data.id)}
      >
        Delete
      </button>
    </div>
  );
};

export default Rule;


Next, create a component for displaying any status or error messages.

import React from "react";

const ErrorMessage = ({ error, styleType }) => {
  const errorDetails = () => {
    if (error.details) {
      return error.details.map(detail => <p key={detail}>{detail}</p>);
    } else if (error.detail) {
      return <p key={error.detail}>{error.detail}</p>;
    }
  };

  const errorType = () => {
    if (error.type) {
      return (
        <em>
          See
          <a href={error.type} target="_blank" rel="noopener noreferrer">
            {" "}
            Twitter documentation{" "}
          </a>
          for further details.
        </em>
      );
    }
  };

  return (
    <div className={`ui message ${styleType}`}>
      <div className="header">{error.title}</div>
      {errorDetails()}
      {errorType()}
    </div>
  );
};

export default ErrorMessage;


Finally, create a component to display a loading indicator during any pending API calls.

import React from "react";

const Spinner = () => {
  return (
    <div>
      <div className="ui active centered large inline loader"></div>
    </div>
  );
};

export default Spinner;

The final step is to proxy requests from your client to your backend server. To do this, from within your src/ directory, create a new file called “setupProxy.js” and add the following code.

const { createProxyMiddleware } = require("http-proxy-middleware");

// This proxy redirects requests to /api endpoints to
// the Express server running on port 3001.
module.exports = function (app) {
  app.use(
    ["/api"],
    createProxyMiddleware({
      target: "http://localhost:3001",
    })
  );
};


You can now start up both the server and client by going to the project root directory and typing the following.

After this command completes, your default web browser should automatically launch and navigate to http://localhost:3000. You will then be taken to the rules management section of the app.

Now that you have an app in place to listen for any kind of Tweet you want, let’s walk through some real-life examples of how this app can be used such as

Learning about personal finance and savings

For each of the examples and accompanying rules listed below, you can navigate to the rules section of the app and simply copy and paste the rule into the input field to add it to your stream. Once the rule has been added, it will take  effect within seconds and only Tweets matching the criteria of that rule will be sent to you.

In this first example, let’s say you are interested in finding remote developer job 

openings. To surface these kinds of Tweets, you can use the following rule.

(developer OR engineer) remote (context:66.961961812492148736 OR context:66.850073441055133696)

To understand what this rule is doing, you can break it down into two parts. The keywords part and the Tweet annotations part.

The keywords part of the rule will match Tweets containing the keywords “developer” or “engineer” and the keyword “remote”. These keywords alone will certainly match Tweets containing remote developer job postings, but it will also match on irrelevant Tweets. For example, the Tweet below will match this rule.

https://twitter.com/EraserFarm/status/1220013392766947332

Since this is not the desired Tweet, you need to take this rule a step further. Though this rule matches irrelevant Tweets, it is also successful in matching Tweets with actual job postings. For example, the rule you have so far will also result in matching the Tweet below.

https://twitter.com/plataformatec/status/1225460423761317888

Tweet annotations: Using context operators

The challenge you have now is, though you are receiving Tweets containing job postings you will still have to go through irrelevant Tweets. If only there was a way to only match on Tweets that contain job postings as best as possible. Are there operators you can use in your rule that only match these relevant Tweets? 

Fortunately, this is where the power of Tweet annotations comes in. Take a closer look at an example Tweet object payload that was sent for this Tweet on your filtered stream endpoint. Within the payload, the nested “context_annotations” field is present. Context annotations are delivered as a “context_annotations” field in the payload. These annotations are inferred based on the Tweet text and result in the domain and/or entity labels, which can be used to discover Tweets on topics that may have been previously difficult to surface. Note, that these fields will only be included if data is present since not all Tweets will contain this data. 

      "context_annotations": [
        {
          "domain": {
            "id": "66",
            "name": "Interests and Hobbies Category",
            "description": "A grouping of interests and hobbies entities, like Novelty Food or Destinations"
          },
          "entity": {
            "id": 961961812492148736,
            "name": "Recruitment",
            "description": "Recruitment"
          }
        },
        {
          "domain": {
            "id": "66",
            "name": "Interests and Hobbies Category",
            "description": "A grouping of interests and hobbies entities, like Novelty Food or Destinations"
          },
          "entity": {
            "id": "850073441055133696",
            "name": "Job search",
            "description": "Job search"
          }
        }
      ],

To match on the domain and entity ids within the context_annotations field, you can use the “context” operator. The “context” operator can be used to instruct your filtered stream endpoint to match on Tweets containing specific domain and entity names. Here’s what that would look like.

(context:66.961961812492148736 OR context:66.850073441055133696)

The first endpoints of the new Twitter API are out and waiting for you to explore and build apps with in order to understand what’s happening in the public conversation.

This tutorial will walk you through how to build your own real-time Tweet streaming app using the filtered stream endpoints and Tweet annotations to listen for Tweets based on your own topics of interest.

Recent Posts