We’re taking a break from our weekly round up of noteworthy apps to share a post written by Mads Hartmann, the Engineering Manager for our platform team. From time to time, we’ll post articles written by Glitch employees that give you a sneak peek into some of the behind-the-scenes work that goes into making Glitch go and grow. 

Over the past year, Mads has written a multi-part series about observability at Glitch and the tools and processes we’ve implemented to make the site more reliable. The most recent post goes into a deep dive on SLO-based alerts - challenges with our previous on-caller system and an excerpt of our internal tech spec. There’s also a getting started guide that’s a good primer if you’re not familiar with SLOs or want to learn more about what goes into their implementation.

“We’re hoping that by implementing SLOs - and alerting on them - we’ll be able to improve communication during incidents, reduce the toil on on-callers, and help improve our reliability in a way that’s meaningful to our users.”

Read the full post here. And if you want to work alongside talented and thoughtful engineers like Mads, we’re hiring. 

Adopting SLO-based alerting on Glitch

Throughout the election season, we saw data from a small number of states captivate the attention of a global audience. Social media feeds were a constant stream of numbers and maps; it was nearly impossible to make sense of it all in real-time. Instead of getting overwhelmed, Daniel Flanagan created Too Close to Call, a Glitch project that does all the marginal math needed to follow close races in competitive states. I reached out to Daniel, a recent college graduate and freelance visual effects artist, to talk about his process and the aftermath of creating a popular app during, and for, such an energizing national moment.

Jenn: With every election there are lots of maps and data–I, for one, often get overwhelmed. What drove you to create your own app among all that noise?

Daniel: That feeling of data overload is exactly what drove me to make this app.  I knew the election was going to come down to a handful of states, but once early counts confirmed as much, I, and I’m quoting myself here, “was annoyed with constantly searching” a whole map of the country when only a few states were relevant.

Add to that the fact that all these maps would only tell you which way the state was leaning and maybe a percentage/vote count if you selected a specific state, it was a pain to find the information that was truly relevant.  I wanted to know how close each state’s race was: 200 votes or 200,000 votes?

I do appreciate the work that goes into all those maps, I follow a few of their developers on Twitter and I think they knocked it out of the park.  My little app is not possible without all their work and the ease with which those maps convey the state of the race at a glance is incredibly valuable.  In this case I just wanted a more concise version of only the data I found relevant.

You mentioned in the support forum that you hold a degree in political science! Do you often build apps that are politics related, or was this a special opportunity for you?

I did yes. Technically it's a minor in political science but I definitely got more out of my poli sci classes than any others.  To be honest, I don’t build all that many apps in the first place, this sort of user-facing development is a hobby more than anything else.

I build things around stuff that interests me, and to say that I was interested in this election would be an understatement.  The challenging thing with anything political is the fact that everyone comes to it with their own politics.  I’m not looking to make something that drives people apart, or even something that affirms someone’s beliefs; the interesting projects to me are the ones that provide the opportunity to bring understanding and context to an area, as Malcolm Gladwell would say, otherwise overlooked or misunderstood.

This was a special opportunity for numerous reasons, chiefly because it allowed me to channel my election energy into something productive.  I was also excited to be working with real-time live data, that’s not something I get to do often.

How long was the process from getting the idea to sharing it with the world?

I had the idea as I was scrolling through Twitter that morning and set up the project on Glitch shortly thereafter.  I didn’t get to start in earnest until lunch around noon, and my work on this project was sporadic throughout the day as I was working on, well, work.  I’d say it probably got 4-5 hours of dedicated attention throughout the day. I took a look at my commit history and it looks like the first commit was at 10am, the UI was done by about 2pm, I made a commit noted “Functional Prototype” just before 5pm, and I shared it on Twitter an hour later at 6:03pm. So from commit to tweet we’re looking at 8 hours and 3 minutes. 

What made Glitch the place you chose to build and share this project?

For any project or idea that requires a web server, Glitch is always my first choice.  I don’t quite understand how Glitch is able to provide so much for free but I’m not complaining.  Glitch containers allow for websocket connections, terminal access, and persistent storage, all while abstracting the infrastructure away from me. Aside from the technical, I feel like my kind of app development, small, one-off experiences fits so well within the Glitch community.  I’ve come across so many awesome small projects on Glitch and being a part of that is something that’s hard to find on the internet.

Did you know how you were going to build the app when you got started or did those decisions come to you as you were coding? For instance, did you go into the process knowing you’d be using WSJ data?

I didn’t start this app until I had the Wall Street Journal data source, which I didn’t even find myself, it came across my Twitter feed courtesy of Paul Miller. I knew when I started this was going to be a Flask based Python app because I’m super familiar with that setup and how it interacts with Glitch.  I built out the UI first, and comparing what it looks like now to a sketch in my notebook, I can confidently say that I built exactly what I set out to build.

Updating the data went through a few design ideas:  First I thought I’d have the data be filled out server-side and sent to the client using Flask’s built-in Jinja templates.  The biggest benefit to that approach was not having to write JavaScript.  JavaScript is a great language and it powers a lot of what I do, but I really don’t like writing it.

Eventually I came to my senses and decided the flexibility of being able to update the data without a full page reload was worthwhile.  I did that with a Flask endpoint and a fairly simple client JavaScript file.  Ironically I built in this flexibility and didn’t take advantage of it until I started getting some feedback on the app.

Once you shared the app, it really took off! How was it received, and did you need to make changes to the app in response to the reception? Was there anything you would have done differently given more planning and time?

It was so exciting, and a little frightening, to see this app take off.  It was a crash course in social media virality for me; I tweeted it to my 81 followers and to date it’s reached 3,000 unique users totaling more than 7,000 page views.

The reception was awesome, it was great to see so many people using it, and it turned out a lot of people were looking for something to cut out all the noise for them.  One of my best friends sent me a photo of the app on his company’s office flat screen and told me how the app flickered as they refreshed it.  Conveniently, I had built in the flexibility to have the app update the data automatically, I just hadn’t implemented it.  That’s where the two minute refresh countdown came from.

In response to comments on Twitter I added the electoral vote counts, the map color changes, and the minimize maps toggle.  I also added PWA (Progressive Web App) capabilities after I kept losing the tab on my phone.

I enjoyed the time pressure, knowing that at any moment it could become irrelevant was a motivator to get it done.  The one thing I wish I did was cache the data and show some sort of trend line from it.  Even something as simple as showing which direction the vote margin was trending would have been awesome.  There’s a similar app another group of developers made that was brought to my attention by people commenting on my app on Twitter that I think does that really well.

This week I’ve been a part of an awesome Glitch ✕ Twitter collaboration on an in-browser Jeopardy board.  I adapted an old API of mine that allows a frontend developed by Patrick Weaver and Trent Willis to pull clues from previously aired episodes of Jeopardy.  You can check out my API, the gameboard, and the Twitter thread.

Other than that I’ve got a lot of ideas but no concrete plans as of yet.  Maybe something similar to this app for the upcoming Georgia Senate runoff elections. Although answering these questions for the Glitch blog is making me think that maybe I want to have my own blog 👀.

Thanks to Daniel, who should most definitely start his own blog, for telling his story and for building Too Close to Call on Glitch! If you build or see an app on Glitch that you want us to feature, tell us about it by emailing community@glitch.com.

Throughout the election season, we saw data from a small number of states captivate the attention of a global audience. Social media feeds were a constant stream of numbers and maps; it was nearly impossible to make sense of it all in real-time. Instead of getting overwhelmed, Daniel Flanagan created Too Close to Call, a Glitch project that does all the marginal math needed to follow close races. 

I reached out to Daniel, a recent college graduate and freelance visual effects artist, to talk about his process and the aftermath of creating a popular app during and for such an energizing national moment.

Too Close to Call: A conversation with creator Daniel Flanagan

Last week we showed how our community has been using WebXR to create new worlds and tell stories in 3D. There’s a lot to be said, though, about how much you can create with only two dimensions. The Glitch community has not only been showing what can be done with a little bit of JavaScript and map tiles, but they’re changing the way we think about maps entirely! These projects don’t just help us get to a place, but they tell compelling stories as well.

Let’s take a look at some of our recent favorites.

Carta Marina, a map of fantastical Nordic sea creatures

Creator @mapsmania has been making some of the coolest interactive maps ever! Some are location-based like their interactive history of the London tube, but some like ~carta-marina guide us through art and history in a really unique way. Every map enthusiast would enjoy their blog, Maps Mania.

Waking and Sleeping, a world map of wakedness

This real-time map of users tweeting “good morning” and “good night” across the world does what most maps don’t do: remind us that we’re not alone. Its creator, creative technologist @bookwormgirl910, describes the project as “a reflection on world interconnectedness and our daily waves and flares of activity and wakedness across the globe.” I recommend you check out the about page of the project to learn how it works and what limitations Katherine faced around languages and geotagging with the Twitter API in particular.

Supertall Vacancies, a story of NYC housing in the shadows of luxury vacancies

This interactive map by @apr2136 and collaborators at Columbia’s Points Unknown takes a look at both commercial and residential real estate in Manhattan and vacancy trends from recent years, and even gives a literal voice to doormen who have witnessed the impact of luxury housing surplus firsthand. 

If you have a story to tell, it’s clear from the above projects that using maps is a great way to get your readers interested. And with Glitch, you can have something running in seconds–you can remix all of the above awesome map projects and more in our collection of storytelling maps. The collection also includes some apps that can get you started with the JavaScript libraries that these creators used for their own projects:

Leaflet.js, a lightweight and open source library for creating apps that are mobile friendly. We created this Guide to Leaflet including remixable apps to get you up and running.

Mapbox, a location data platform that lets you build powerful location-based apps on the web. They’re on Glitch at @mapbox and have posted some great apps to get you started!

We can’t wait to see what you create and what stories you tell–we’d love for you to let us know about them! If you have Glitch projects you’d like us to feature, tweet them to us @glitch or email community@glitch.com! ✨

The project featured in this post's header image is ~lhh by @darzhz

Worlds to Explore: Storytelling with Maps

As many of us are spending more time at home and looking at the same screen before, during and after work, a change of scenery is often welcome. Lucky for us, the Glitch community has been producing incredibly compelling immersive web experiences that take us to other worlds, tell stories, and inspire and teach us to create our own. 

Here are some of the virtual spaces I’ve found myself getting lost in over the past couple of weeks. Happy exploring! 

Groad Tour, a glimpse of Portland then and now 

Creator Nathan Backous takes us all on a virtual tour of all-ages venues around Portland, Oregon where the band JonnyX and the Groadies played at and have since closed. "Some of the venues captured in this project are separated over 20 years from their previous purpose. Some of the buildings remain relatively unchanged, some are drastically different, some are entirely gone,” writes Backous. Each “stop” contains flyers, signs, and photos from the band giving viewers a true trip down memory lane. 

The Homer Metaverse, for Simpsons fans everywhere

Something that creator Fabio Cortés and I have in common is that our first exposure to VR was an episode of The Simpsons from 1995 where Homer finds himself in a mysterious 3D world behind a book shelf. This Glitch app is a playful throwback to the series.

Jurassic Warp, a multi-media exhibit (Note: it auto-plays sound on page load)

This is an incredible A-frame art piece created by artists KT Duffy and María Luisa Sanín Peña. It’s just one of many examples of artists converting their work in response to recent gallery closings. According to New City Art, Duffy’s original exhibition was planned for Ground Level Platform in Chicago, but then the project was moved online to Glitch!

These are just a handful of projects to check out; you can find them and more in our Immersive Web Experiences collection. If you’re interested creating your own immersive web experiences, there are a lot of great resources to get you started. Here are just a few:

Check out our WebVR starter kit, which includes short videos and app examples that introduce you to WebVR and guides you through creating your own virtual solar system in A-frame.

For a guided introduction to A-frame, check out their free interactive course, A-Frame School, and you can remix any of their starter apps here on Glitch.

I’m a big fan of the Immersive Web Weekly newsletter, published by the W3C’s Immersive Web working group. Subscribers get a weekly email curated with news, events, art and projects related to the immersive web, and it’s a great way to learn more and be inspired by this community!

We can’t wait to see what you create, and we can’t wait to help you show your projects off! If you have Glitch projects you’d like us to feature, tweet them to us @glitch or email community@glitch.com! ✨

The project featured in this post's header image is ~earthy-course by @adarosecanon.

﻿﻿As many of us are spending more time at home and looking at the same screen before, during and after work, a change of scenery is often welcome. Lucky for us, the Glitch community has been producing incredibly compelling immersive web experiences that take us to other worlds, tell stories, and inspire and teach us to create our own.

Worlds to Explore: Immersive Web Experiences

Over the summer we announced fast and free static sites on Glitch, providing you a way to create and share simple webpages that never go to sleep. Static sites are great for ambitious client-side apps, landing pages for educational demos, compelling artworks from code - basically any project that needs only HTML, CSS and client-side JavaScript to come to life! Some of you have reached out asking how to turn your full-stack JavaScript apps into static sites so you can save on Project Hours and keep your projects awake for free, so I’m going to show you how to get started. 

Glitch distinguishes between a static and full-stack site by watching for any of three simple configuration files:  package.json,  requirements.txt and glitch.json. Those two last files are for configuring apps built in other experimental-to-Glitch languages; for this tutorial I’m going to focus on full-stack JavaScript apps using Node.js, which are configured in package.json. If you’re not familiar with the anatomy of a Node app, you can check out our free zine on Making Web Apps with Node!

To get started, you’ll need a full-stack app that’s running all of its important code on the client-side. For this tutorial I created ~full-stack-to-static, whose server-side code only contains routes for where the client side files are. 

The first thing I do is remove the server-side files and see what breaks and troubleshoot from there. So, I remove package.json, letting Glitch know that this should be a static site. Then I remove server.js, and .env since they will not be run without package.json.

Going to the app view, I see that something looks very wrong. Instead of my index.html content, I see a file listing like this:

This is because my index.html file is still in a“views” folder. Renaming“views/index.html” to“index.html” moves that file in the root of my project and fixes that.

I can now see index.html rendering but I now get errors when trying to load my stylesheet and JavaScript. This is because they are inside a“public” folder but my index.html references them as if they were in the root of my project - something my server-side code had handled before. I can fix this either by updating links in index.html to add the“public/” directory, or by renaming“public/style.css” and“public/script.js” to“style.css” and“script.js”. In this example, I did the latter and now my static site is looking great. Here's an embed of the final result:

Now I have a project that does basically the same thing it was doing before, but it stays awake and it doesn’t count towards my Project Hour limits. I recommend you take a look at your Project Dashboard and see if you have any full-stack projects using up Project Hours that could be refactored to static sites! And for all your projects that need to be full-stack apps, just remember as always that Glitch gives you 1000 free Project Hours each month to run your apps, and subscribers can lift those limits if you need even more.

The project featured in this post's header image is ~iris-cockroach by @potch. Have a project you want us to feature? Tweet them to us @glitch or email community@glitch.com!

Over the summer we announced fast and free static sites on Glitch, providing you a way to create and share simple webpages that never go to sleep. Static sites are great for ambitious client-side apps, landing pages for educational demos, compelling artworks from code - basically any project that needs only HTML, CSS and client-side JavaScript to come to life! Some of you have reached out asking how to turn your full-stack JavaScript apps into static sites so you can save on Project Hours and keep your projects awake for free, so I’m going to show you how to get started...

Going from Full-stack to Static

The code for this tutorial is available on GitHub. You can also check out the live demo of the app on Glitch

The first endpoints of the new Twitter API are out and waiting for you to explore and build apps with in order to understand what’s happening in the public conversation.

This tutorial will walk you through how to build your own real-time Tweet streaming app using the filtered stream endpoints and Tweet annotations to listen for Tweets based on your own topics of interest.

After building the app, you will learn about how it can be applied to some real-life examples to inspire you to get started such as.

Discovering new music videos: Imagine a dashboard that lets you see the music videos being shared across Twitter

Finding remote developer job openings: Imagine creating a remote developer job listings app with remote developer job openings being shared across Twitter

Learning about personal finance and savings: Surface public conversations about personal finance and savings happening on Twitter

To get started here’s what you will need:

You must have an approved developer account, and have activated the new developer portal experience. If you don’t have one already, you can apply for one. Access is available with active keys and tokens for a developer App that is attached to a Project created in the developer portal. 

A Bearer Token from your App in the Twitter developer portal.

Npm (This is automatically installed with Node. Make sure you have npm 5.2 or higher.)

First, install Node.js. Check out the Downloads section from Node’s website and download the source code or installer of your choice. Alternatively, if you are running on a Mac you can install the Node package using the Brew package manager

Open a terminal window and bootstrap your React app using create-react-app by using npx. 

npx create-react-app real-time-tweet-streamer

After create-react-app has finished executing, change to the newly created real-time-tweet-streamer directory and replace the scripts block in your package.json with the following script block in your package.json. These lines will provide a command shortcut to concurrently run your client and server backend code in development or production as needed.

"scripts": {
  "start": "npm run development",
  "development": "NODE_ENV=development concurrently --kill-others \"npm run client\" \"npm run server\"",
  "production": "npm run build && NODE_ENV=production npm run server",
  "client": "react-scripts start",
  "server": "node server/server.js",
  "build": "react-scripts build",
  "test": "react-scripts test",
  "eject": "react-scripts eject"
},

After updating the scripts section, your package.json should now look as follows.

Next, remove all files within the src/ subdirectory. 

Then, create a new file within the src/ subdirectory called index.js. The code for this file will be as follows.

import React from "react";
import ReactDOM from "react-dom";
import App from "./components/App";

ReactDOM.render(<App />, document.querySelector("#root"));

Connecting to the filtered stream endpoints requires you to authenticate using a bearer token from your app in the Twitter developer portal. To utilize your bearer token, you will need to have the following environment variable set. You can do so by issuing the following command in your terminal window assuming you are using bash as your shell. Replace <YOUR BEARER TOKEN HERE>, including the left and right angle brackets, with your bearer token.

export TWITTER_BEARER_TOKEN=<YOUR BEARER TOKEN HERE>

First, you will need to get started with implementing the Node server, which will be responsible for making the actual requests to the Twitter API. This Node server will serve as a proxy between your browser-based React client and the Twitter API. On your Node server, you will need to create API endpoints that connect to the filtered stream endpoints. In turn, requests from your React client will be proxied through to your local Node server. 

Before you go any further, cd to the project root directory and install the following dependencies

npm install concurrently express body-parser util request http socket.io path http-proxy-middleware request react-router-dom axios socket.io-client react-twitter-embed

Next, while still within your project root directory, create a new subdirectory called “server” and a new file within that subdirectory called “server.js”. 

This source code file will contain all of your backend logic for connecting to and receiving Tweets from the filtered stream endpoint. The contents of your server.js file will be as follows.

Filtering Tweets on the filtered stream endpoint using rules

Filtered stream has two endpoints, a streaming endpoint, to receive data and a rules endpoint that is used to create and delete rules. The filtered stream endpoints require you to define search queries called rules, in order for it to know what kind of Tweets to send to you. Rules allow you to narrow down to only the Tweets you are looking for by using a set of operators. You will see some example use cases and corresponding rules you can use later once you finish building the app.

The other filtered stream endpoint is the streaming endpoint, which uses a simple GET connection. Once a connection is established, Tweets are delivered in JSON format through a persistent HTTP streaming connection. You will only receive Tweets matching your rules while connected to the stream. 

The next step is to work on the following React components

App.js - The parent component that be will, in turn, render all other components

NavBar.js - Displays the navigation bar for navigating between the Tweet feed and managing rules

Tweet.js - Displays a Tweet on the page 

TweetFeed.js - Renders multiple Tweet components at once in a “feed” like fashion

Rule.js - Renders an individual rule on your stream

RuleList.js - Renders multiple Rule components and displays an input field to add a rule

ErrorMessage.js - Renders any status or error messages to the screen

Spinner.js - Renders a loading indicator for any pending API calls

Now you will need to get started with creating the React components. Under your /src subdirectory, create a directory called “components”. The source code files above will be stored in this new directory. First, create the parent most component of the application. This component will be responsible for rendering all other components.

import React from "react";
import { BrowserRouter, Route } from "react-router-dom";

import Navbar from "./Navbar";
import TweetFeed from "./TweetFeed";
import RuleList from "./RuleList";

class App extends React.Component {
  render() {
    return (
      <div className="ui container">
        <div className="introduction"></div>

        <h1 className="ui header">
          <div className="content">
            Real Time Tweet Streamer
            <div className="sub header">Powered by Twitter data</div>
          </div>
        </h1>

        <div className="ui container">
          <BrowserRouter>
            <Navbar />
            <Route exact path="/" component={RuleList} />
            <Route exact path="/rules" component={RuleList} />
            <Route exact path="/tweets" component={TweetFeed} />
          </BrowserRouter>
        </div>
      </div>
    );
  }
}

export default App;


Next, create the component for the navigation bar.

import React from "react";
import { NavLink } from "react-router-dom";

const Navbar = () => {
  return (
    <div className="ui two item menu">
      <NavLink to="/tweets" className="item" target="_blank">
        New Tweets
      </NavLink>
      <NavLink to="/rules" className="item" target="_blank">
        Manage Rules
      </NavLink>
    </div>
  );
};

export default Navbar;


Next, create the parent component for rendering all job listings.

import React, { useEffect, useReducer } from "react";
import Tweet from "./Tweet";
import socketIOClient from "socket.io-client";
import ErrorMessage from "./ErrorMessage";
import Spinner from "./Spinner";

const reducer = (state, action) => {
  switch (action.type) {
    case "add_tweet":
      return {
        ...state,
        tweets: [action.payload, ...state.tweets],
        error: null,
        isWaiting: false,
        errors: [],
      };
    case "show_error":
      return { ...state, error: action.payload, isWaiting: false };
    case "add_errors":
      return { ...state, errors: action.payload, isWaiting: false };
    case "update_waiting":
      return { ...state, error: null, isWaiting: true };
    default:
      return state;
  }
};

const TweetFeed = () => {
  const initialState = {
    tweets: [],
    error: {},
    isWaiting: true,
  };

  const [state, dispatch] = useReducer(reducer, initialState);
  const { tweets, error, isWaiting } = state;

  const streamTweets = () => {
    let socket;

    if (process.env.NODE_ENV === "development") {
      socket = socketIOClient("http://localhost:3001/");
    } else {
      socket = socketIOClient("/");
    }

    socket.on("connect", () => {});
    socket.on("tweet", (json) => {
      if (json.data) {
        dispatch({ type: "add_tweet", payload: json });
      }
    });
    socket.on("heartbeat", (data) => {
      dispatch({ type: "update_waiting" });
    });
    socket.on("error", (data) => {
      dispatch({ type: "show_error", payload: data });
    });
    socket.on("authError", (data) => {
      console.log("data =>", data);
      dispatch({ type: "add_errors", payload: [data] });
    });
  };

  const reconnectMessage = () => {
    const message = {
      title: "Reconnecting",
      detail: "Please wait while we reconnect to the stream.",
    };

    if (error && error.detail) {
      return (
        <div>
          <ErrorMessage key={error.title} error={error} styleType="warning" />
          <ErrorMessage
            key={message.title}
            error={message}
            styleType="success"
          />
          <Spinner />
        </div>
      );
    }
  };

  const errorMessage = () => {
    const { errors } = state;

    if (errors && errors.length > 0) {
      return errors.map((error) => (
        <ErrorMessage key={error.title} error={error} styleType="negative" />
      ));
    }
  };

  const waitingMessage = () => {
    const message = {
      title: "Still working",
      detail: "Waiting for new Tweets to be posted",
    };

    if (isWaiting) {
      return (
        <React.Fragment>
          <div>
            <ErrorMessage
              key={message.title}
              error={message}
              styleType="success"
            />
          </div>
          <Spinner />
        </React.Fragment>
      );
    }
  };

  useEffect(() => {
    streamTweets();
  }, []);

  const showTweets = () => {
    if (tweets.length > 0) {
      return (
        <React.Fragment>
          {tweets.map((tweet) => (
            <Tweet key={tweet.data.id} json={tweet} />
          ))}
        </React.Fragment>
      );
    }
  };

  return (
    <div>
      {reconnectMessage()}
      {errorMessage()}
      {waitingMessage()}
      {showTweets()}
    </div>
  );
};

export default TweetFeed;

Next, create the child component for the previous component that renders an individual Tweet. 

import React from "react";
import { TwitterTweetEmbed } from "react-twitter-embed";

const Tweet = ({ json }) => {
  const { id } = json.data;

  const options = {
    cards: "hidden",
    align: "center",
    width: "550",
    conversation: "none",
  };

  return <TwitterTweetEmbed options={options} tweetId={id} />;
};

export default Tweet;


Next, create the component responsible for rendering all the rules on our stream as well as displaying input controls for creating new rules. In this case, we will only be using one rule.

import React, { useEffect, useReducer } from "react";
import axios from "axios";
import Rule from "./Rule";
import ErrorMessage from "./ErrorMessage";
import Spinner from "./Spinner";

const reducer = (state, action) => {
  switch (action.type) {
    case "show_rules":
      return { ...state, rules: action.payload, newRule: "" };
    case "add_rule":
      return {
        ...state,
        rules: [...state.rules, ...action.payload],
        newRule: "",
        errors: [],
      };
    case "add_errors":
      return { ...state, rules: state.rules, errors: action.payload };
    case "delete_rule":
      return {
        ...state,
        rules: [...state.rules.filter((rule) => rule.id !== action.payload)],
      };
    case "rule_changed":
      return { ...state, newRule: action.payload };
    case "change_loading_status":
      return { ...state, isLoading: action.payload };
    default:
      return state;
  }
};

const RuleList = () => {
  const initialState = { rules: [], newRule: "", isLoading: false, errors: [] };
  const [state, dispatch] = useReducer(reducer, initialState);
  const exampleRule = "from:twitterdev has:links";
  const ruleMeaning = `This example rule will match Tweets posted by 
     TwtterDev containing links`;
  const operatorsURL = "https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/integrate/build-a-rule";

  const rulesURL = "/api/rules";

  const createRule = async (e) => {
    e.preventDefault();
    const payload = { add: [{ value: state.newRule }] };

    dispatch({ type: "change_loading_status", payload: true });
    try {
      const response = await axios.post(rulesURL, payload);
      if (response.data.body.errors)
        dispatch({ type: "add_errors", payload: response.data.body.errors });
      else {
        dispatch({ type: "add_rule", payload: response.data.body.data });
      }
      dispatch({ type: "change_loading_status", payload: false });
    } catch (e) {
      dispatch({
        type: "add_errors",
        payload: [{ detail: e.message }],
      });
      dispatch({ type: "change_loading_status", payload: false });
    }
  };

  const deleteRule = async (id) => {
    const payload = { delete: { ids: [id] } };
    dispatch({ type: "change_loading_status", payload: true });
    await axios.post(rulesURL, payload);
    dispatch({ type: "delete_rule", payload: id });
    dispatch({ type: "change_loading_status", payload: false });
  };

  const errors = () => {
    const { errors } = state;

    if (errors && errors.length > 0) {
      return errors.map((error) => (
        <ErrorMessage key={error.title} error={error} styleType="negative" />
      ));
    }
  };

  const rules = () => {
    const { isLoading, rules } = state;

    const message = {
      title: "No rules present",
      details: [
        `There are currently no rules on this stream. Start by adding the rule 
        below.`,
        exampleRule,
        ruleMeaning,
      ],
      type: operatorsURL,
    };

    if (!isLoading) {
      if (rules && rules.length > 0) {
        return rules.map((rule) => (
          <Rule
            key={rule.id}
            data={rule}
            onRuleDelete={(id) => deleteRule(id)}
          />
        ));
      } else {
        return (
          <ErrorMessage
            key={message.title}
            error={message}
            styleType="warning"
          />
        );
      }
    } else {
      return <Spinner />;
    }
  };

  useEffect(() => {
    (async () => {
      dispatch({ type: "change_loading_status", payload: true });

      try {
        const response = await axios.get(rulesURL);
        const { data: payload = [] } = response.data.body;
        dispatch({
          type: "show_rules",
          payload,
        });
      } catch (e) {
        dispatch({ type: "add_errors", payload: [e.response.data] });
      }

      dispatch({ type: "change_loading_status", payload: false });
    })();
  }, []);

  return (
    <div>
      <form onSubmit={(e) => createRule(e)}>
        <div className="ui fluid action input">
          <input
            type="text"
            autoFocus={true}
            value={state.newRule}
            onChange={(e) =>
              dispatch({ type: "rule_changed", payload: e.target.value })
            }
          />
          <button type="submit" className="ui primary button">
            Add Rule
          </button>
        </div>
        {errors()}
        {rules()}
      </form>
    </div>
  );
};

export default RuleList;


Next, create the child component of RuleList.js responsible for displaying a single rule and deleting a rule.

import React from "react";

export const Rule = ({ data, onRuleDelete }) => {
  return (
    <div className="ui segment">
      <p>{data.value}</p>
      <div className="ui label">tag: {data.tag}</div>
      <button
        className="ui right floated negative button"
        onClick={() => onRuleDelete(data.id)}
      >
        Delete
      </button>
    </div>
  );
};

export default Rule;


Next, create a component for displaying any status or error messages.

import React from "react";

const ErrorMessage = ({ error, styleType }) => {
  const errorDetails = () => {
    if (error.details) {
      return error.details.map(detail => <p key={detail}>{detail}</p>);
    } else if (error.detail) {
      return <p key={error.detail}>{error.detail}</p>;
    }
  };

  const errorType = () => {
    if (error.type) {
      return (
        <em>
          See
          <a href={error.type} target="_blank" rel="noopener noreferrer">
            {" "}
            Twitter documentation{" "}
          </a>
          for further details.
        </em>
      );
    }
  };

  return (
    <div className={`ui message ${styleType}`}>
      <div className="header">{error.title}</div>
      {errorDetails()}
      {errorType()}
    </div>
  );
};

export default ErrorMessage;


Finally, create a component to display a loading indicator during any pending API calls.

import React from "react";

const Spinner = () => {
  return (
    <div>
      <div className="ui active centered large inline loader"></div>
    </div>
  );
};

export default Spinner;

The final step is to proxy requests from your client to your backend server. To do this, from within your src/ directory, create a new file called “setupProxy.js” and add the following code.

const { createProxyMiddleware } = require("http-proxy-middleware");

// This proxy redirects requests to /api endpoints to
// the Express server running on port 3001.
module.exports = function (app) {
  app.use(
    ["/api"],
    createProxyMiddleware({
      target: "http://localhost:3001",
    })
  );
};


You can now start up both the server and client by going to the project root directory and typing the following.

After this command completes, your default web browser should automatically launch and navigate to http://localhost:3000. You will then be taken to the rules management section of the app.

Now that you have an app in place to listen for any kind of Tweet you want, let’s walk through some real-life examples of how this app can be used such as

Learning about personal finance and savings

For each of the examples and accompanying rules listed below, you can navigate to the rules section of the app and simply copy and paste the rule into the input field to add it to your stream. Once the rule has been added, it will take  effect within seconds and only Tweets matching the criteria of that rule will be sent to you.

In this first example, let’s say you are interested in finding remote developer job 

openings. To surface these kinds of Tweets, you can use the following rule.

(developer OR engineer) remote (context:66.961961812492148736 OR context:66.850073441055133696)

To understand what this rule is doing, you can break it down into two parts. The keywords part and the Tweet annotations part.

The keywords part of the rule will match Tweets containing the keywords “developer” or “engineer” and the keyword “remote”. These keywords alone will certainly match Tweets containing remote developer job postings, but it will also match on irrelevant Tweets. For example, the Tweet below will match this rule.

https://twitter.com/EraserFarm/status/1220013392766947332

Since this is not the desired Tweet, you need to take this rule a step further. Though this rule matches irrelevant Tweets, it is also successful in matching Tweets with actual job postings. For example, the rule you have so far will also result in matching the Tweet below.

https://twitter.com/plataformatec/status/1225460423761317888

Tweet annotations: Using context operators

The challenge you have now is, though you are receiving Tweets containing job postings you will still have to go through irrelevant Tweets. If only there was a way to only match on Tweets that contain job postings as best as possible. Are there operators you can use in your rule that only match these relevant Tweets? 

Fortunately, this is where the power of Tweet annotations comes in. Take a closer look at an example Tweet object payload that was sent for this Tweet on your filtered stream endpoint. Within the payload, the nested “context_annotations” field is present. Context annotations are delivered as a “context_annotations” field in the payload. These annotations are inferred based on the Tweet text and result in the domain and/or entity labels, which can be used to discover Tweets on topics that may have been previously difficult to surface. Note, that these fields will only be included if data is present since not all Tweets will contain this data. 

      "context_annotations": [
        {
          "domain": {
            "id": "66",
            "name": "Interests and Hobbies Category",
            "description": "A grouping of interests and hobbies entities, like Novelty Food or Destinations"
          },
          "entity": {
            "id": 961961812492148736,
            "name": "Recruitment",
            "description": "Recruitment"
          }
        },
        {
          "domain": {
            "id": "66",
            "name": "Interests and Hobbies Category",
            "description": "A grouping of interests and hobbies entities, like Novelty Food or Destinations"
          },
          "entity": {
            "id": "850073441055133696",
            "name": "Job search",
            "description": "Job search"
          }
        }
      ],

To match on the domain and entity ids within the context_annotations field, you can use the “context” operator. The “context” operator can be used to instruct your filtered stream endpoint to match on Tweets containing specific domain and entity names. Here’s what that would look like.

(context:66.961961812492148736 OR context:66.850073441055133696)

The operators above follow the format “context:<domain id>.<entity id>”. As seen in the example payload above, the domain id 66 represents the “Interests and Hobbies Category”. The entity ID 961961812492148736 represents the “Recruitment” entity and the entity ID 850073441055133696 represents the “Job search” entity. For a complete list of domains, the Tweet annotations docs contain a table with 50+ domain names.

With the operator explanation out of the way, this 2nd part of the rule matches Tweets containing the entity names “Recruitment” or “Jobs search”.

In summary, taking both parts of this rule together, it will match on Tweets that contain the keywords “developer” or “engineer” and the keyword “remote”, but only if those Tweets also contain the entity names “Recruitment” or “Jobs search”

If you need new music video suggestions, you can start by using a simple rule that matches on Tweets containing the keywords “song” and “YouTube”. You will also want Tweets that actually link out to external videos

Taking a closer look at the payload of this Tweet, you see that it has some annotations on it that can help you match more relevant Tweets. Notice the Annotation with an entity label of “Pop” and a domain name of “Music Genre”

     "context_annotations": [
        {
          "domain": {
            "id": "10",
            "name": "Person",
            "description": "Named people in the world like Nelson Mandela"
          },
          "entity": {
            "id": "871815676998033408",
            "name": "Ally Brooke",
            "description": "Ally Brooke"
          }
        },
        {
          "domain": {
            "id": "54",
            "name": "Musician",
            "description": "A musician in the world, like Adele or Bob Dylan"
          },
          "entity": {
            "id": "871815676998033408",
            "name": "Ally Brooke",
            "description": "Ally Brooke"
          }
        },
        {
          "domain": {
            "id": "55",
            "name": "Music Genre",
            "description": "A category for a musical style, like Pop, Rock, or Rap"
          },
          "entity": {
            "id": "810938279801470977",
            "name": "Pop",
            "description": "Pop"
          }
        }
      ],

To make this rule better and narrow down your Tweets to be even more relevant, you can update your rule as follows.

song youtube has:links context:55.810938279801470977

This will take the original rule you used a step further by narrowing down to only the Tweets that are labeled with the Music Genre domain label and the Pop entity label.

As a final example, let’s say you are interested in learning about personal finance and you can be savvier about your spending and savings. You also only want original Tweets that contain links to online articles to learn more. 

Going through a similar process as you did with earlier examples if you simply add the following rule, only Tweets containing the words “personal”, “finance” and “savings” will be sent to you.

Taking a look at one of the Tweet payloads, the nested annotations contains an entity label about personal finance that will help you narrow down your Tweets to the most relevant ones

 "annotations": {
          "context": [
            {
              "context_domain_id": 66,
              "context_domain_id_str": "66",
              "context_domain_name": "Interests and Hobbies Category",
              "context_domain_description": "A grouping of interests and hobbies entities, like Novelty Food or Destinations",
              "context_entity_id": 847888632711061504,
              "context_entity_id_str": "847888632711061504",
              "context_entity_name": "Personal finance",
              "context_entity_description": "Personal finance"
            },

Using the context operator to match on Tweets containing this label, you can revise your rule to look as follows

context:66.847888632711061504 has:links -is:retweet savings

Using the filtered stream endpoints to stream publicly available Tweets to your server and annotations, you created an app to more easily surface Tweets around a topic of interest. The filtered stream endpoint gives you the haystack of data in the form of Tweets and the Tweet annotations help you find the needle in that haystack. 

Have you found other interesting ways to use this app? Follow me on Twitter and send me a Tweet to let me know. I used several libraries beyond the Twitter API to make this tutorial, but you may have different needs and requirements and should evaluate whether those tools are right for you.

The first endpoints of the new Twitter API are out and waiting for you to explore and build apps with in order to understand what’s happening in the public conversation.

This tutorial will walk you through how to build your own real-time Tweet streaming app using the filtered stream endpoints and Tweet annotations to listen for Tweets based on your own topics of interest.

Building an app to stream Tweets in real-time using the Twitter API

When we think about our plans for Glitch and what we need to build, it’s usually in service of two things:

How can we empower creators to build the web and make the experience fun and accessible?

How can we help as many people as possible and grow our business in the process?  

Today we’re excited to share some changes that make the Glitch experience better while also offering more transparency about our free tier and paid membership. 

Since Day 1, we’ve always put Glitch projects to sleep no matter how big or small. It was just a general rule that we enforced in order to keep our servers happy and run more efficiently. Over the last month or so, we’ve made some significant changes to our back-end, which now means  your Static Sites never go to sleep — even for free users!

We pride ourselves on being a great full-stack solution for web apps, but we also know that so many community members just want a quick and easy solution to create and share simple web pages. From students sharing out a resume for the first time to local Pet Adoption centers, we’ve seen a whole range of pure HTML, CSS, and JS goodness. 

If you have static sites on Glitch, check your projects view and you’ll see a little label that says “Static Site”.

Important note: If you’re not seeing the “Static Site” label and you know the project is static, just wake it up or hop into the editor and you should be all set!  

Back in April we launched Boosted Apps, which is a paid upgrade on Glitch. Members who opt-in get up to 5 boosted projects, which keeps them on and also increases rate limits, disk space, and memory. 

One question that kept coming up for some users is, “How do I know if paying for Glitch is right for me?” Here’s our official take. 

If you want to build something quickly or you're just tinkering, get started for free. (We don’t even make you sign-in if you wanna give the editor a spin; just create an account if you need to save your work.) 

If you want to make a lot of stuff, but don’t need projects to run all the time, the free experience might still be enough. Here’s where today’s update comes in. 

We’re introducing Project Hours to give you greater insight into the resources that your projects are using without the typical complexity associated with cloud hosting. We don’t want you to have to think about things like compute, transfer, dynos, and droplets. All you need to know is every month you’ll have 1,000 hours for your projects to run and they’ll reset back to 1,000 on the first of each month. This new policy will go into effect August 1, but you can see and monitor your July hours in your dashboard now. 

If you find that the 1,000 hours isn’t enough, that’s when you should upgrade. You’ll get the 1,000 hours plus the 5 boosted slots, which you can rotate projects in and out of.

In short, there’s a lot that comes with the free Glitch experience. As of today, you’re getting static sites hosted for free without going to sleep. You’re getting a full-stack container and in-browser editor that’s simple enough for beginners, but powerful enough for professionals. And you’re getting 1,000 hours of free hosting time. 

If and when you need more magic under the hood, upgrade and support the community. In the meantime, we’ll continue working to make the Glitch experience the most reliable, intuitive, and friendly place to build the web.

Fast and free static sites on Glitch and new Project Hours

One month ago we talked to you about the increased number of incidents that Glitch was experiencing at the time, and how we’re committed to keeping Glitch reliable so that you can focus on creating and sharing:

“In the coming weeks, we’re going to be making significant changes that will have a big impact on uptime and stability. On top of that, we’re working on improving your experience with managing projects on Glitch.”

We wanted to give you an update on how things are going and a peek at more good things to come. 

We have seen a significant improvement in uptime over the past few weeks. The platform is experiencing fewer and shorter incidents. This is a huge win for us and wouldn't be possible if our community hadn’t helped us make some key decisions like reducing the impact of pinging services on the site. Thank you so much for sticking with us through the turbulence. 

Also, special shout out to our Support Forum regulars. You all have been super helpful and welcoming to newcomers asking about service incidents, policy changes and general questions about code and Glitch!

Something we quietly launched earlier in the month was a more straight-forward way to manage what you make on Glitch. If you’re logged in, you can try it at https://glitch.com/dashboard, and you can also get to it by clicking “Your Projects” from your user menu on the top-right of the homepage. 

We want building and sharing apps to be frictionless and intuitive, which is why we've always shielded you from having to know the complexities around how projects are run. That said, we've heard many of you say you want more information about your projects and greater control over how you manage them. This new tool is a step towards creating that experience — and it’s going to keep getting better.

A lot of the changes coming up are all in service of empowering creators like you to quickly and easily host projects, but also give you the stability and control you might expect from more conventional hosting providers. 

We have to strike a balance between keeping things simple and free, but also give everyone the tools and transparency to manage more resources if they need them. Launching boosted apps was a big first step in this direction, but it’s just a start.  

Hopefully you’ll find our next set of updates a nice enhancement to the free Glitch experience and also an upgrade for those of you looking for more power. 

An Update On How We’ve Been Making Glitch More Reliable

We’re doing a ton of work behind the scenes to make our infrastructure more stable.  As we mentioned in last week’s post:

One of our big challenges is dealing with third-party services that ping Glitch to keep apps awake. Not all of these services are as reliable as we’d like, and sometimes when there are problems they can cause as much load on our systems as a malicious attack would. We’re not banning the use of pinging (or “uptime”) services on Glitch right now, and we’d like to not have to. 

We did some more investigating and started to temporarily block traffic from ping services to help reduce the load on our servers. The temporary block allowed us to see just how intense a strain they put on the platform. 