Marcos Placona Blog

Programming, technology and the taming of the web.

Category: General Techie Stuff (page 1 of 5)

All of the general techie talk that will not fit on any of the specific categories

UK Top 40 albums & singles JSON

Reading time: 1 – 2 minutes

top-40-albuns-singles

So I had this idea for a little application and wanted to get the UK’s Top40 singles to use in it. I started by writing something that would scrape Radio 1′s Top 40 chart and return me a list of songs since I couldn’t find any feeds that would give me that.

I then thought this could be of use to somebody else, so turned it into a little service (built using Ruby and Sinatra) that returns a JSON object with all the singles and some useful information about the number of weeks it’s been there, how much has it moved, and which direction (up or down) it’s gone.

On the root of it, it returns the chart date as the current date and time, and the retrieval date to indicate which date it’s been last retrieved. I am caching the feed to play nice with Radio 1, so I’m only making one HTTP request a day to their website.

Check it out at http://top40.placona.co.uk/

Also feel free to fork it, and collaborate by adding your country’s top 40

My NodeJS app development experience

Reading time: 3 – 5 minutes

So, I decided to take a punt at writing a NodeJS application. Not a big application, or anything that would get me slashdoted, but an application that would help me understand the language, and give me a taste of what it is so many people are talking a bout. I have worked with JavaScript for quite a while now, and NodeJS seems like one of those “just right” languages, that you pick up in an afternoon and come to love it after writing a few lines of code. I read a few tutorials on the subject. And then decided to dive straight into building a small application that uses GitHub’s API. There were three things I wanted to take out of it:

Installing everything was painless, but I have to admit getting NodeJS to build on a Raspberry PI wasn’t the most straight forward thing I’ve ever done. Not because it wouldn’t work, but because building it from source took forever. Just to clarify, I didn’t need to run it on a Raspberry PI, but just wanted to prove myself I could. This mini project had 5 elements to it:

Three from which you don’t even need to install (as they are npm modules). Just by adding them as dependencies on the package.js file, they get installed into your project, so deploying is made easier (in theory), as you don’t actually need to install anything other than NodeJS.

Now you might be asking yourself why I say “in theory” when taking about deployment. The fact is that deploying a NodeJS application is a pain, since all you end up doing is making [your preferred webserver here] proxy all the calls to your local application. Why? you may be asking…

The fact that each NodeJS application you create establishes its own server, means anything will clash with your already installed webserver, since you can only have one webserver per TCP port. So if you want to host all your applications as well as any new node project you come up with, then you will end up taking this route. It’s not terrible, but just means you now need to support yet another thing. There are tools out there to help you with such thing, but let’s not digress.

Building the application was fairly painless, and using Jade was the big highlight for me. For a while now I had looked into templating engines, and really wanted to delve into it a little more. I had previously looked at jQuery Templates, Ext and Dust.js, but Jade is totally different, in a way that actually pleases the eyes (except when it doesn’t :-)).

Did I become a NodeJs expert? No way!

Will I keep looking and building sexy applications with it? Heck yeah!

Enough about me though.

Have a look at the application I wrote here, and make sure to send me pull requests if you would like to improve it.

Managing your dotfiles the right way

Reading time: 3 – 4 minutes

It’s no secret that on the UNIX world, dotfiles play a very important part when it comes to making your terminal look good. Be it on Linux, be it on a Mac. Dotfiles are there so you can configure your favourite software to look just the way you like it.

I especially use dotfiles to customize the look on my terminal, and to manage bundles I use with Vim. One thing that normally annoys me, is the fact that whenever I rebuild my machine (or build a new one) I need to copy over my dotfiles, and obviously make sure they are kept up-to-date on all my devices when I change something.

I’ve heard about people adding their dotfiles to GitHub, and even noticed GitHub themselves encourage you to do the same. I decided to give it a go, and will describe here what you need to do in order to have your dotfiles stored there, and most importantly, how to quickly load them up on any other computers boxes you may have.

Start by creating a folder called “dotfiles” on your home directory, and move all your dotfiles into it.

In the example above, I’m only covering my vim and bash dotfiles. You can cover as many as you like by simply moving your files into the dotfiles directory.

Now it’s time to create your install script also under our ~/dotfiles directory. You should use this script every time you want to install your dotfiles on a given machine. So let’s open vim (or your favourite text editor) and create the following file:

The file is pretty simple, as it contains a list of files which you want to have copied, and a loop to put them on the right location and create symlinks on your home directory.

Now, it’s really important that you name this file correctly, as you want to be able to execute it. And after having created it, we need to give it execute permissions, as we want to be able to run it

If you run this file now, you should see the following happening on your screen:

Good. We’re ready to push this to GitHub, so we never have to go through this process again.

Go ahead and create  a new repository on GitHub called dotfiles.

Now, back to your terminal and run the following commands:

From top to bottom we’ve done the following:

  1. navigated to our newly created directory
  2. turned it into a local git repository
  3. added all the files it contained into our local Git repository
  4. committed all the added files into the new local repository
  5. hooked up our local git repository with the GitHub repository
  6. pushed our files to the remote (GitHub) repository

If everything went OK, you should now be able to browse your remote repository (mine is https://github.com/mplacona/dotfiles) and see all your dotfiles (as well as your installer) listed there.

Now, the beauty of it, is the fact that whenever you want your dotfiles in any other box, you can simply do the following:

All your settings will be copied and applied.

As previously mentioned, my dotfiles are in https://github.com/mplacona/dotfiles, so fell free to fork it, modify and maybe even collaborate.

Making your JMeter tests timeless.

Reading time: 2 – 4 minutes

At work, we use Apache JMeter for load testing applications or API’s we build.

JMeter is an amazing open source tool with an even more amazing community behind it, so if you’ve never used it, you surely are missing on a great piece of software.

Anyway, one thing that used to annoy me a bit, was the fact that when creating tests for API’s, I’d sometimes need to pass in dates.

So imagine the following case:

You’re testing that your API can create a new book in your bookstore for a specified date, but the business rules imply that you can only create a new book with a release date in the future.

If I had hard coded this to a date in the near future, I would then need to update it at some point as to always make sure my dates really were in the future.

I could also put a date very far in the future, but my business rules again dictate that I can only pass in dates up to 60 days in advance, so I’d end up having to update my tests in about two months or so.

Come the user defined variables:

With JMeter, you can create “user defined variables” (add -> config element -> user defined variables) and assign values to it. Those variables can be created with JavaScript, so you could potentially create a variable in there, that would always default to today’s date plus a given number of days (in this example anything that didn’t exceed 60 days)

JMeter - User Defined Variables

The code for your user defined variable would look like this:

What it does is simply create a new date, add 60 days to it, and then format it as “yyyy-mm-dd”. You could change the number of days and format easily by moving things around.

A sample request would then look something like this:

<book>
	<name>The Magic Book</name>
	<isbn>0123456789</isbn>
	<category>children</category>
	<release>${date}</release>
</book>

Which would generate a request as such:

PUT http://my.api.com/v1/books/add
PUT data:
<?xml version="1.0" encoding="UTF-8"?>
<book>
	<name>The Magic Book</name>
	<isbn>0123456789</isbn>
	<category>children</category>
	<release>2012-11-01</release>
</book>

And with that, your tests become timeless, as you can run then whenever you like without having to worry about updating dates.

What breaks WP Super Cache

Reading time: 2 – 3 minutes

Disclaimer: This blog post is not meant to be timeless, meaning it will lose its relevancy as soon as the stuff I’ll mention below gets updated. I will do my best to keep up with the updates, and delete this blog post as soon as it becomes irrelevant.

I’ve recently noticed my WordPress blog (latest version) caching functionality was broken due to the error:

The pages do not match! Timestamps differ or were not found!

This would make my blog not cache at all.

I then started on a “Google mission” to see if I could find reasons to why this was happening. As far as I could see many people were having this same error and it seemed to have started inexplicably.

I found some people were reporting that the error started to happen after they’d installed JavaScript minifying plugins.

I do have one of those plugins, but after disabling it, my cache still wasn’t working. However, this convinced me the problem could be related to some other plugin I had installed.

I decided to go on and disable every single plugin in my blog, except for WP Super Cache. I then confirmed this was the problem, as my caching started to work again.

There were 20 plugins installed, and I decided to activate all the ones that’s were less likely to cause a conflict; such as administrative tools, email management and form processing.

That left me with about 10 plugins to investigate.

I started to enable one by one, and found that what was causing my cache to fail, was an advertising plugin I had, that I used to display textlinkads.

More precisely, TextLinkAds on versions 3.9.7 and 3.9.8. It was then easy to find a post reporting this exact same problem. The only way around it is disabling the plugin. In my case, it was a no-brainer choice between giving my blog a massive performance boost, or making a few bucks showing sponsored ads, but I guess you’ll have to make a call on that yourself.

Like I said, I presume the authors of TextLinkAds will fix this incompatibility soon, and I’ll remove this blog post as soon as they do so. In the meantime, I think it’s worth having it here so other people can benefit from it.

Older posts