Category Archives: Web

HTTP Request Inspectors

Someone wrote in to let me know that an older project of mine which provides a simple webservice echo test was referencing some out of date projects. My HTTP test service doesn’t use any of those other services, but I’ve updated the blog post description to point to some new options for comparison purposes.

The original hosted version of RequestBin (at requestb.in) is no longer live. This was run by Runscope at the time and the source code is still up on GitHub (Runscope/requestbin). You can see their the status change:

We have discontinued the publicly hosted version of RequestBin due to ongoing abuse that made it very difficult to keep the site up reliably. Please see instructions below for setting up your own self-hosted instance.

Here’s the commit with the awesome (but sad) commit message of :( when they turned it off. Some good comments there from fans of the service.

The good news is with the source code still available, people can still use the service themselves (Heroku and Docker instructions are included) and the adventurous ones can run live services using the same. Here are a few I found with some quick searching (but have not personally used):

Tweetfave Support for Longer Tweets

Tonight I spent a couple hours troubleshooting a problem with my Tweetfave service and handling of links. Luckily I discovered it was a simple matter of keeping up with Twitter’s API changes. This service has been running so smoothly and the API has actually been pretty stable. I needed to dust off my PHP skills and dig in to track down and adapt to an important change.

Background

Tweetfave is a free service which monitors the tweets you mark as favorites (now referred to as “likes”), then sends them to you by email. Currently the system stats show over 250 users have tried the service, recording just under 400,000 tweets!

One of the handy features in Tweetfave is the “un-shortening” of the short URLs embedded in the tweet. Rather than showing a generic “http://t.co/xyz” link, the software will reveal the original URL.

Problem

This has all been working fine, but recently I noticed something strange where the URL decoding resulted in a link back to the original tweet. Instead we should be seeing the the links within the tweet.

Here’s an example with the original tweet which doesn’t have any links, but does include an embedded image which should have a Twitter image URL:

Truebeck tweet screenshot
Example tweet with image

However when it’s processed by Tweetfave, the resulting email snippet shows a link back to https://twitter.com/i/web/status/831963043508596736 which is the original tweet:

Truebeck tweetfave email snippet
Incorrect Tweetfave email snippet

Solution

It took a bit of trial and error, debug logging, and Google searching to find the culprit: the support for more than 140 characters in tweets. When Twitter added that support, they wanted to retain compatibility. Any API call which returned tweet text was still limited to 140 characters, with the link back to the tweet to read the rest.

It looks like the announcements came out in May 2016:

They were a little fuzzy about when the actual changes would happen, but from my logs I think it was around October 2016.

In the end the change was quite simple:

  1. Include the parameters extended_tweet=true in the API requests
  2. Retrieve the original tweet text from the full_text rather than text response field

With those changes in place everything works again. For example this tweet from TheNextWeb includes both an article link and an image:

TNW tweet screenshot
Example tweet with image and link

Once decoded and sent by Tweetfave, the result included decoded links for both the article and the picture:

TNW Tweetfave email snippet
Corrected Tweetfave email snippet

Tech Advent Calendars – 2016

City Advent Calendar 2012 by brickset on Flickr

It’s that time of the year again and I’m happy to see Advent calendars for many tech communities are still going strong. As in past years (2011, 2012, 2013, 2014, and for some reason skipped 2015), I’ve gathered a few here that I’ll be following this year:

In years past I also created a combined feed through Yahoo! Pipes, but sadly that service was shut down in 2015.

Luckily there is still a bit of RSS and feed infrastructure out there, including the aptly-named RSS Mix. Here’s a combined RSS feed with all of the above calendars: http://www.rssmix.com/u/8215936/rss.xml

Update: The RSS Mix feed was sometimes not working correctly, so I also created a mix using MailChimp’s ChimpFeedr service: http://mix.chimpfeedr.com/af982-Tech-Advent-2016

Update #2: From Perl Weekly I just learned about Advent Planet which combines all of the above advent calendars and more into one mega-advent!

Happy Holidays!

Tech Advent Calendars – 2014

Update: For the latest, check out Tech Advent Calendars – 2016

It’s that time of the year again – Advent calendars for many tech communities. As in past years (2011, 2012, 2013), I’ve gathered a few here that should be interesting:
* Perf Planet Advent (performance) – Feed
* 24ways Advent (web design/development) – Feed
* Perl Advent (Perl language) – Feed
* Java Advent (Java language) – Feed
* UXMas (UX for everyone) – Feed
* SysAdvent (Sysadmin) – Feed I have a combined RSS feed (created with Yahoo! Pipes) that picks up all of these advent calendars:

http://feeds.feedburner.com/TechAdventCalendars. (Yahoo Pipe source).

Creating Print Stylesheets for Other Sites

Creating a print stylesheet for your website is a nice optimization that your visitors will appreciate, especially if your site features articles that are longer or are technical in nature. Print stylesheets don’t take too much effort to create and maintain over time along with your site CSS.

But what if you’d like to have print stylesheets on someone else’s website?

Here’s an approach I’ve used to hack together simple print stylesheets that I’ve sent to other site owners as a contribution. This works very well for personal blogs and open source projects, both of whom are usually open to contributions.

Make a Local Copy

The first step is to make a local copy of a representative page from the site. For example, if the site is a blog, pick a good sample post (not the home page or an index page).

These steps require wget which already exists on most Mac/Linux systems, or can be installed for Windows.

  1. Download the sample page with wget: wget -p --convert-links URL
  2. If the downloaded file doesn’t have extension .html, rename it (e.g. if the download was index.php, rename it to index.html)
  3. Open the .html file in a local browser and make sure everything looks okay; some dynamic content such as ads may not appear correctly, but that’s okay
  4. Edit the .html file and after all the other CSS includes, add reference to a new local file: <link rel="stylesheet" href="print.css" media="all" />
  5. Create the print.css file and start with something that will be obvious like: p {color:red;}
  6. Refresh the .html file in the browser and make sure the local print.css is being picked up

Create Print Stylesheet

Now we’ll go through and build up print.css to hide and adjust content sections as needed to create a good looking printout. For this step I usually use Chrome or Firefox with their built-in Inspect Element features. You can make iterative changes to the print.css and refresh in the browser.

Hide Sections

Find all sections/divs that are not the content. This will typically be the header, footer, sidebar, ads, comments, and so on. You’ll need to adjust your CSS selectors depending on the content, starting with the most general and working down as needed.

For each section identified, set it to display:none. For example:

header, footer, div#leftnav {
    display: none;
}

Adjust Widths

Most site designs will have a hierarchy of <div> elements which result in the main content being narrower than the full page. We want to adjust that out to 100% to be more appropriate for a printout. A good starting point is the following:

div#content, #main {
    width: auto;
    max-width: 100%;
    float: none !important;
}

Adjust Fonts

You might need to adjust the font settings, particularly the size. There is also the question of serif vs sans-serif and which works better for print. (You choose.) You may also need to reduce font size and margins for heading elements <h1>, <h2>, etc. if they are being printed too big.

For example:

div#content, #main {
    font: 11pt Georgia, "Times New Roman", Times, serif;
    line-height: 1.3;
}

Optional

Depending on the site content, here are some other things to consider:

  • Hide any embedded video objects like YouTube
  • Hide comments/discussion
  • Set color to black on white background
  • Add URLs after <a> links

Contribute Back to Site Owner

Once you have your print.css file ready to go, I suggest adding comments to clarify the different changes you made. That way, the recipient can better understand what you’ve done, and make selective changes if they want.

Send the file to the site owner as a .txt file, or via a service like pastebin. They can use your contribution either as a standalone print stylesheet (<link rel="stylesheet" href="print.css" media="print" />) or directly inside an existing stylesheet at the end (@media print { ... }).

Good luck. If you use this technique and are successful in creating a print stylesheet for someone else’s site, let me know!