Category Archives: Web

Tweetfave Support for Longer Tweets

Tonight I spent a couple hours troubleshooting a problem with my Tweetfave service and handling of links. Luckily I discovered it was a simple matter of keeping up with Twitter’s API changes. This service has been running so smoothly and the API has actually been pretty stable. I needed to dust off my PHP skills and dig in to track down and adapt to an important change.

Background

Tweetfave is a free service which monitors the tweets you mark as favorites (now referred to as “likes”), then sends them to you by email. Currently the system stats show over 250 users have tried the service, recording just under 400,000 tweets!

One of the handy features in Tweetfave is the “un-shortening” of the short URLs embedded in the tweet. Rather than showing a generic “http://t.co/xyz” link, the software will reveal the original URL.

Problem

This has all been working fine, but recently I noticed something strange where the URL decoding resulted in a link back to the original tweet. Instead we should be seeing the the links within the tweet.

Here’s an example with the original tweet which doesn’t have any links, but does include an embedded image which should have a Twitter image URL:

Truebeck tweet screenshot
Example tweet with image

However when it’s processed by Tweetfave, the resulting email snippet shows a link back to https://twitter.com/i/web/status/831963043508596736 which is the original tweet:

Truebeck tweetfave email snippet
Incorrect Tweetfave email snippet

Solution

It took a bit of trial and error, debug logging, and Google searching to find the culprit: the support for more than 140 characters in tweets. When Twitter added that support, they wanted to retain compatibility. Any API call which returned tweet text was still limited to 140 characters, with the link back to the tweet to read the rest.

It looks like the announcements came out in May 2016:

They were a little fuzzy about when the actual changes would happen, but from my logs I think it was around October 2016.

In the end the change was quite simple:

  1. Include the parameters extended_tweet=true in the API requests
  2. Retrieve the original tweet text from the full_text rather than text response field

With those changes in place everything works again. For example this tweet from TheNextWeb includes both an article link and an image:

TNW tweet screenshot
Example tweet with image and link

Once decoded and sent by Tweetfave, the result included decoded links for both the article and the picture:

TNW Tweetfave email snippet
Corrected Tweetfave email snippet

Tech Advent Calendars – 2016

City Advent Calendar 2012 by brickset on Flickr

It’s that time of the year again and I’m happy to see Advent calendars for many tech communities are still going strong. As in past years (2011, 2012, 2013, 2014, and for some reason skipped 2015), I’ve gathered a few here that I’ll be following this year:

In years past I also created a combined feed through Yahoo! Pipes, but sadly that service was shut down in 2015.

Luckily there is still a bit of RSS and feed infrastructure out there, including the aptly-named RSS Mix. Here’s a combined RSS feed with all of the above calendars: http://www.rssmix.com/u/8215936/rss.xml

Update: The RSS Mix feed was sometimes not working correctly, so I also created a mix using MailChimp’s ChimpFeedr service: http://mix.chimpfeedr.com/af982-Tech-Advent-2016

Update #2: From Perl Weekly I just learned about Advent Planet which combines all of the above advent calendars and more into one mega-advent!

Happy Holidays!

Recommended Tech Podcasts

Microphone photo

Here’s what I’m currently listening to when I can. They’ve been piling up a little bit now that my commute is so short, but last week I had a round-trip drive from San Jose to Fresno so really got caught up :) See last year’s updates at
Best Technical Podcasts.

.NET Rocks! [rss]

Hosted by Carl Franklin and Richard Campbell, this is probably the podcast I’ve listened to the longest. As their site says, the shows “range from introductory information to hardcore geekiness.” In particular I really like their “geek out” episodes, most recently: Supersonic Aircraft Geek Out

Startups For the Rest of Us [rss]

Continues to be a favorite of mine with lots of discussion around bootstrapped startups, with updates from hosts Rob Walling and Mike Taber and occasional interviews.

Hanselminutes [rss]

Still one of my favorites. Scott is really a great interviewer and has a real variety of guests & topics. Recent interesting episode (with a sweet tag line): Failure – “What if you fake it and don’t make it?” with Kronda Adair

Giant Robots Smashing into other Giant Robots Podcast [rss]

Ben Orenstein and Chris Toomey talk about design, code and marketing decisions for a couple of thoughtbot SaaS products.

Bootstrapped Web [rss]

Brian Casel and Jordan Gal give weekly updates on their own “bootstrapped” web projects, with occasional guest interviews.

The Changelog [rss]

All open source, all the time. Recent (awesome!) episode: The Changelog #200: JavaScript and Robots with Raquel Vélez (rockbot)

RunAs Radio [rss]

Hosted by Richard Campbell (also co-host .Net Rocks!), this podcast is more IT-focused rather than developer-focused. Recent interesting episode: The Infrastructure Release Pipeline with Steve Murawski and Michael Greene

Software Engineering Daily [rss]

I found this one when I heard that Tim Berglund would be talking about Cassandra and have stuck around for a nice variety of technical interviews.

Photo credit:
heil pr40 in studio ozdorp by Peet Sneekes (Flickr)

Twitter Favorites RSS Feeds

Twitter dropped RSS feeds in early 2013 (Mashable) and never had direct support for RSS feeds from favorites. I still use Twitter favorites as a bookmark or “read it later” service for myself and have been running the Tweetfave service for over 2 years now. Tying these ideas together, I’ve just rolled out Twitter favorites RSS feed support in Tweetfave.

How it Works

Tweetfave periodically scans your account (using the official API) and summarizes all favorited tweets in an email. Now the system will also update a feed which can be used with other systems that accept standard RSS feeds.

The RSS feed will be created with each tweet contained in an item element with the following fields set:

  • title – text version of the tweet
  • description – text version of the tweet, plus a link to the original tweet
  • content:encoded – HTML version of the tweet, plus a link to the original tweet
  • link – the first link mentioned in the tweet (see Notes below)
  • guid – same as link
  • pubDate – date/time of the original tweet

Here’s a screenshot of my favorites feed shown in Firefox:

Screenshot of favorites RSS feed in Firefox
Favorites Feed in Firefox

Setup Instructions

For existing Tweetfave users:

  1. Visit the Tweetfave login page
  2. Sign in to Twitter (if needed) and approve the Tweetfave App
  3. Copy the link for “Twitter favorites RSS feed”

For new Tweetfave users:

  1. Visit the Tweetfave login page
  2. Sign in to Twitter (if needed) and approve the Tweetfave App
  3. Enter your email and click Submit
  4. Mark a couple of tweets as favorite to get started
  5. Within a couple hours you should get your first email from Tweetfave
  6. Now return using the “existing users” steps above to grab your RSS feed link

What to Do

So what can be done now that you have an RSS feed of your Twitter favorites? I like to use IFTTT which is an awesome application for connecting different services together. (For a great overview, see their About IFTTT page).

I’ve set up my recipes where the source trigger is a new item in the RSS feed and the target is a service like Instapaper, Paper, or Evernote. You can configure the fields and formatting sent to various services, giving you lots of different options.

Screenshot of IFTTT example recipes
IFTTT Recipe Examples

Notes

Here are a few notes and caveats to keep in mind:

  • If you’re a new Tweetfave user, you need to favorite at least one tweet first, and wait for the first email before your feed link is created
  • Your RSS feed link is somewhat obfuscated so that people can’t guess it. It’s also only shown to you after login, so you can still keep your favorites stream private.
  • If a favorited tweet has multiple links, you will have multiple RSS items (because each one can only have one link). This seems like a good compromise even at the expense of some extra entries.
  • The time between favoriting a tweet and it appearing in your feed is not instant; Tweetfave currently scans everything at 2-hour intervals, then sends emails and updates the feeds.
  • If you disable your Tweetfave account, the RSS feed file will still be online but won’t be updated. If you’d like the file removed, just email me.

Feedback

I’d love to hear feedback from anyone trying this out, including any interesting use cases people come up with!

Yahoo! Pipes Closing

Just A Pipe Dream

I was sad to see this week that Yahoo! Pipes will be closing soon. Pipes launched in early 2007, so it had a pretty good 8+ year run.

People created some pretty crazy mashups with Pipes. For me the simple ability to combine RSS feeds with some simple logic was very useful (for example: 2014 Tech Advent Calendars). I also recently created some Stack Overflow feed tools for work which will have to be recreated in Python or something similar.

The news was part of a broader product update from Yahoo: Q2 2015 Progress Report On Our Product Prioritization. Also included was the news that Yahoo! Maps will be closing down at the end of this month.

I worked at Yahoo for almost 7 years and the phrase “Product Prioritization” became kind of a good-news/bad-news thing towards the end. It always sounds good (it’s logical to focus resources on the most important products and sites), but if the closure affected the site that you work, that’s not as much fun. (In the end that was the final result for me and much of my team on the Yahoo Developer Network.)

Photo Credit: Just A Pipe Dream by Paul B (Flickr)