Posts by dan

Behind the Scenes at QueryTree: The Anatomy of a Web Based Data Tool

QueryTree is a web based drag and drop tool for doing simple data analysis. Users drag and connect tools in a chain to first load in data, then manipulate that data (filter, group, sort etc.) and then visualise that data on a chart or map. I started building QueryTree by myself in between freelance projects […]

Exporting all Sheets on a Spreadsheet to a Single CSV

If you have a spreadsheet with multiple sheets/tabs containing similar tables of data; and you want to export the whole lot to a single CSV, then this VBA macro should help: Sub ExportAllSheetsToSingleCSV() ‘The file to write to outputFile = “C:\Users\dan\output.csv” f = FreeFile() Dim headerLine As String Open outputFile For Output As f For […]

Parsing Large CSV Blobs on Google App Engine

When parsing a blob on Google App Engine using the Python CSV library, the simplest approach is to pass the BlobReader straight into the CSV reader. However, unlike when with opening a normal file, there is no option to handle universal newline characters. In order to handle all the different kinds of newline characters, the […]

BigData: Size isn’t Important, it’s What You Do With it That Counts

Lets say you run a medium sized business. You employ 1000 people and have 5000 customers. Heard about “BigData”? Think you should probably be doing some of that too? Well, don’t start building that Hardoop cluster just yet, because you probably don’t need it… To illustrate, let’s imagine all 1000 of your employees spend 40 […]

Free Speech, the Press and #Leveson

It’s supposed to work like this: You can say or write anything, no matter who it hurts If what you say is wrong, someone else is free to say so and expose you If you’re exposed as being wrong, people stop listening to you This is the basic idea of free speech. Many people have […]

Vim Tips

To search for the word under the cursor, type “*”.

To search for some text you’ve already yanked, type “/”, then “Ctrl+R”, then “0”.

To search for some text you’ve already yanked into register “a”, type “/”, then “Ctrl+R”, then “a”.

To search for some text you’ve copied into the clipboard, type “/”, then “Ctrl+R”, then “+”

Handling Method Parameter Default Values Using Moq

We at Contigo often use a pattern where methods with multiple nullable parameters retrieve collections of entities from a data store a bit like this:

IEnumerable GetThingsFromDatabase(int? intFilter = null, bool? boolFilter = null, stringFilter = null)
{
...
}

These methods treat a null value as “no filtering required” so they can be called with only those filters that are relevant to the calling code. In this

What Are You Optimising For?

Very few software designs/architectures are completely and objectively bad. Most designs optimise for something. Here’s a [probably incomplete] list of things that you could optimise for in software architecture:

  • Developer Time Now
    When we say we’re doing something “quick and dirty”, or that we are taking out “Technical Debt”, then we are optimising the development time now.
  • Developer Time Later
    This is what people traditionally mean by “good” code. A

Follow People on Hacker News with hn_comment_follow

I often find the comments on Hacker News are fantastic and there are certain users who’s opinion I always value. To that end I’ve created a python script to help follow what my favourite people are saying on Hacker News: hn_comment_follow. It’s on GitHub in case others would like to fork it.

To call it, invoke the script like this:   
    python hn_comment_follow.py pg patio11 d4nt

Five Software Architect Antipatterns

I believe that all software design comes down to trade-offs, and the only way for software architects to get to the right decisions on those trade-offs is for them to have a very broad and senior role within their organisation. I’ve observed some people in the industry who are called architects but, for one reason or another, are not in a position to design software well. Here are five types