-
My first pop at showing the relative UK opening box office figures, and subsequent weeks, for films going back to 1998. I drew the graphic using Canvas, with data scraped out of IMDb. This links to an image, as the original is quite slow and unwieldy, but I plan to optimise that next.
The data is copyright Nielsen, so I purposely haven't put too much information into the chart, just a couple of film title so you can see what's going on. I think it makes quite an attractive image on its own, although I'd like to add a money scale, try and colour code the seasons of the year a little more accurately, perhaps add in some useful dates like the Oscars and Baftas etc. I'd really like to be able to identify more of the films but I suppose I'll have to work out the copyright issues, or see if anyone has any freely available data to use.
February at paulcarvill.com, the home of Paul Carvill on the web 2009 at paulcarvill.com, the home of Paul Carvill on the web
paulcarvill.com
Hi, I'm Paul Carvill and I'm a web developer. I am Head of Interface Development at LBi, Europe's largest digital agency.
I also like walking, cooking, Bollywood and rock 'n' roll.
Archive for February, 2009
links for 2009-02-15
Sunday, February 15th, 2009
Guardian TechTalk with OpenStreetMap — Steve Coast
Sunday, February 15th, 2009These are some notes from memory:
We had Steve Coast from OpenStreetMap at a Guardian TechTalk on Friday 13th February 2009. Steve Coast now lives in San Francisco, and works with a couple of developers there, plus a couple more in Russia. He talked about the OpenStreetMap Foundation, of which he is the Chairman. OpenStreetMap has no employees, Steve Coast spends most of his time getting one volunteer to interact with another.
Google Maps, Windows Live maps and other big online maps use costly data bought from big mapping companies like NAVTEQ and Tele Atlas, who in turn also purchse mapping info from people like Ordnance Survey. You mostly cannot reuse or derive products from this data. They map using big trucks driving around collecting GPS data, and that data is updated every 18 months – almost exactly the lifespan of your average TomTom, so the purchase of a new one goes directly to fund a new round of mapping. They can’t map where the trucks can’t go. And they’re not interested in OpenStreetMaps free data.
OpenStreetMap operates on a Creative Commons Attribution-Share Alike license – you are free to use the data, edit and update it, as long as you attribute OpenStreetMap as the original source of the data and also share your edits back into the community.
Dealing with conflict – in more than one sense of the word. Maps, as everyone knows, are power. Areas of conflict have a tendency for people to redraw the borders. Steve Coast plainly has better things to do, and other things to worry about – he basically doesn’t care how people use the service, and only once so far have they had to ban users.
On the subject of China – Steve is fairly resigned. He sees any action on his part to persuade China to allows its citizens to freely access OpenStreetMap as futile. Google bent over backwards to get access to China,
‘and they’re not evil, right?’
he says,
’so what difference can I make’.
His philosophy boils down to ‘I made a thing which makes a map. so either use it to make a map, or don’t. I don’t make money out of it, so what do I care?‘
Aerial magery used includes some provided under federal law by the US Government. Yahoo also kindly allow their aerial imagery to be used. Most people start by tracing this into a map. The Netherlands’ biggest mapping company donated all their Netherlands map data — when they looked at it to check all was working shortly after they donated it, it had been updated and edited so heavily they no longer recognised it!
Mapping parties organise groups of people with small GPS devices to map an area. Handheld device is about the same size as a chunky mobile phone. OpenStreetMap loans the deices out so people can contribute mapping data, or you can use your own devices. Data is then uploaded by computer and converted into bitmapped map tiles.
All mapping data by each user is recorded. In the event of someone copying data illegally by tracing copyrighted maps, OpenStreetMap’s policy is to ‘fail quickly and remove all the offending person’s data.’
Mpping data is incremented every night. The map data file is currently about 5Gb.
Everything is hosted at University College, London, where the project was started. Asked how they felt about hosting it, or if they even knew it was there, Steve Coast said,
‘the right people know it’s there’.
You can map anything – pubs, postboxes, roads, footpaths. He shows an example where the OpenStreetMap data is more current and more accurate than Google’s – university campuses, for example – buildings, footpaths are all mapped, where Google shows just a grey area.
Is currently involved in looking for sponsorship, is willing to talk to anyone. Had their first fundraising drive in 2008 for new hardware, made $17,000 in 2 days.
Steve talked to Ordnance Survey, advised us, ‘don’t invest in them if they ever go public’.
Good graphical presentation on updates around the globe over a year-long period. Uptake follows the pattern of a large increase following mapping party, dropping by about 10% soon afterwards and from there continuing at that flat level. They haven’t generally see nmuch drop off after that anywhere.
When asked several tricky questions, Steve Coast repeats the mantra, ‘you fix it.’ The software is open source – if you want something done, do it, or find someone who can do it for you.
Finally, he mentions FreeThePostcode, a campaign to get a free, public database of UK postcodes to rival the Royal Mail’s paid-for model. There is an iPhone app – the idea is that you upload an accurate GPS point and the correct postcode for that point, which is then added to the database and will eventually be made available freely.
Lots of this information, and more, is available at the OpenStreetMap wiki.
links for 2009-02-14
Saturday, February 14th, 2009-
A Valentine's Day crossword proposal
-
Interview with, and preview of a 'webinar' featuring, Bill Scott, Netflix Director of UI Engineering, on the topic of interface engineering.
-
Proper, French cooking-style, 'rapid professional techniques' for making an omelette. With diagrams. And wonderful subheadings like 'Eggs and How to Beat Them'.
-
Why can't we just stretch chicken wire across jet engine openings to stop birds flying into them? Because it just wouldn't work, apparently.
You’ll miss me when I’m gone: IE6, cross-browser consistency and device independence
Saturday, February 14th, 2009A flurry of IE6 related activity on the web this week coincided with a discussion we are having at The Guardian on the same subject. We have been talking about the relative benefits of keeping website performance in IE6 consistent with that of other browsers, and the disproportionate amount of work this requires on the parts of developers and the QA team. We’ve been trying to figure out better processes to reduce the number of styling bugs in IE6, while not compromising the user experience or the hard work put in by our design team.
It turns out people have surprisingly strong views on cross-browser consistency. For some, IE6 represents much more than just ‘a browser’. It also represents, variously: a large market share; an important group of corporate users; a user’s freedom to choose whichever device she wishes to browse the web. Once you start dropping a browser for technological reasons, the argument goes, you might as well arbitrarily drop support for anything which you consider below par – mobile browsers, text browsers, people with small monitors.
The opposing view says that IE6 is many years old and two versions out of date, a huge security risk and a drain on resources. We shouldn’t be pandering to slow or paranoid IT departments who refuse to upgrade their systems. Anyway no one chooses to use IE6, it is forced upon them by said IT departments.
I’m loath to branch the code to produce a separate version of the site for any reason, be it a device or a browser. But I also see the amount of pain IE6 causes developers, especially when they’re trying to do something fancy with JavaScript, and even more especially trying to do so without using a standard library which might easily provide you with cross-browser methods for doing stuff.
I support IE because I have to. But I do also believe strongly in wide accessibility, through as many devices as possible. We should assume nothing — nothing — about how our users access the web. But I don’t think this is the point here. The point here is adhering web standards, which apply to both code and content. Remember, the content itself — the information — usually isn’t broken. It’s what you’re trying to do with it that’s broken. The CSS and the JavaScript. Go back to Tim Berners-Lee’s 2002 document on universality and device independence for a lesson in what putting stuff on the web is all about. Work with the web, not against it. It’s really good at presenting and sharing text and pictures. But it’s not a magazine layout. Berners-Lee once said,
“Anyone who slaps a ‘this page is best viewed with Browser X’ label on a Web page appears to be yearning for the bad old days, before the Web, when you had very little chance of reading a document written on another computer, another word processor, or another network.”
We can infer from this that a site isn’t ‘best viewed in’ anything: it’s just ‘viewed’, however it might end up. So, yes, your site might look lovely, but if getting it there is so complex that it breaks browsers, or takes up 50% of your development time, then you’re plainly doing something wrong.
Try taking your page back to basics, get rid of the awful advertisement JavaScript and the three different kinds of page tracking, and start paying more than just lip-service to web standards and accessibility. That XHTML doctype declaration you’re using, trying adhering to it. There, it probably works a lot better now, yes?
But ultimately, and as usual, I think the whole issue comes down to a business decision: how much time/money are we spending on development versus how much money that development brings in. It’s a brave person who decides to cut off 25% of their users.
Some points that came up as part of our ongoing discussion:
- Should the design be 100% consistent across all browsers, or would our designers be happy to sacrifice certain style elements? We currently stop a code release if something looks bad in IE6, although we have already made one or two decisions to remove an element from IE6 in order to expedite a code release. In both cases we ran things past the Guardian’s Creative Editor, Mark Porter, before doing so.
- If you want to drop suport for IE6, you have to completely and utterly drop support for it. And in all likelihood never look at it again. Because the next time you do, it will be horrifically broken. Stopping development on that browser doesn’t just mean it won’t get cool new features. It still gets the features, but they won’t be tailored to it, and will break it. That smart Javascript widget you just wrote? That breaks the page in IE6. Some new element you put in with a fixed width and margins? That breaks the page in IE6. You have to cut the cord. Be strong, give it a firm handshake and say goodbye.
- Turns out Microsoft haven’t quite cut the cord yet, though. Microsoft support Windows XP Service Pack 3 as a current product (it shipped in April 2008), and will retire support for it 2 years after the next service pack is released, or at the end of the Windows XP product lifecycle, whichever comes first. IE6, which shipped as a component of XPSP3, continues to have Mainstream Support as part of that product:
- Our current browser usage figures look like this:
- IE 7: 35%
- IE 6: 25%
- Firefox 3: 25%
- Safari: 7%
- Firefox 2: 3%
- Google Chrome: 1.5%
- Opera: 0.5%
- We currently have a problem even testing in IE6, because the corporate build on the PCs we use doesn’t contain it, it has IE7 as standard. And you can’t run IE7 and IE6 concurrently. Ironically, our technical infrastructure is sufficiently advanced that we have difficulty supporting old technology.
That flurry of activity in full:
- http://www.robertnyman.com/2009/02/09/stop-developing-for-internet-explorer-6/
- http://www.456bereastreet.com/archive/200902/no_more_pixel_perfectionism_in_ie_6/
- http://arapehlivanian.com/2009/02/12/its-our-fault-ie6-still-lingers/
- http://www.wait-till-i.com/2009/02/13/to-hell-with-ie6-more-like-to-hell-with-good-enough/
Coming soon: mobile versions of sites…
links for 2009-02-13
Friday, February 13th, 2009-
Puntastic
-
Bespin is Mozilla's web based, open source IDE
-
This theory on how to bail out the banks, with little risk to the taxpayer and a truer valuation of the banks' toxic assets, sounds complicated but also sounds like it Might Just Work.
-
We need Twitter's OAuth implementation out of beta, and soon
-
Extraordinary pictures made using long exposures and torches. What sets these images apart from an infinite number of Bonfire Night snaps is their perfectly realised perspective and the desolate, eerie nature of the background scenery.
links for 2009-02-12
Thursday, February 12th, 2009-
I think Michael Sheen's performance as David Frost might have been a bit hammy and caricatured after watching the original Frost/Nixon videos.
-
NYTimes.com is being ludicrously productive right now. TimesPeople API lets you access registered Times readers' details, comments, recommendations, plus their connections i.e. other Times users associated with them as friends.
The API sounds like it might be quite niche relative to broader existing communities, but it's still useful that they have all that data accessible in-house to make available through the interface. Not too clear on how discovery of user IDs or usernames works — probably need to request these from your user.
links for 2009-02-12
Thursday, February 12th, 2009-
"I can't justify getting nominated for an Oscar or a Grammy, that to me wouldn't mean anything if I don't actually get to speak about this. It's not like I'm trying to sell records, I'm trying to stop the deaths of 350,000 people this month,"
-
Dedicated page with maps and news on the continuing bush fires in Australia.
-
Will charge for BOSS searches for developers making over 10000 requests a day.
links for 2009-02-10
Tuesday, February 10th, 2009-
Incredibly useful, very clearly explained tutorial for using Processing with a plug-in JSON library to visualize some data from the NYTimes.com Article Search API
-
Paul Bradshaw, senior lecturer in Online Journalism, Magazines and New Media and Birmingham City University, is blogging his course materials.
He starts this week with 'Passive-Agressive Newsgathering'
It's a great idea, though it's surprising how simple some of these ideas are. I suppose sitting in front of a computer for a large part of the day makes it easy to take these avenues of communication for granted.
-
A-Rod, a baseball player or the New York Yankees and one of the world's highest paid sportsmen, has admitted taking performance enhancing drugs in the past.
Includes the unbelievable line, when confronted about his flat-out denial of drug-taking to CBS's Katie Couric,
"At the time…I wasn't even being truthful with myself. How am I going to be truthful with Katie or CBS?"
Bravo.
Amazingly, A-Rod has a clause in his conrtact that calls for him to be paid an extra $30 million for reaching each of five milestones — tying each of the top four players on the all-time home run list, then surpassing the current holder, Barry Bonds, recently embroiled in a drugs scandal of his own.
-
Steps 1-4 are the actual coding bit, the rest is getting your app out there.
-
Jeff Bezos: "Our vision is every book, ever printed, in any language, available in less than 60 seconds."
-
Which US states will benefit from Obama's stimulus package
-
Includes some really geeky stuff on the algorithms being used by teams attempting to win the Netflix $1,000,000 prize for improving it's recommendation engine
links for 2009-02-08
Sunday, February 8th, 2009-
Steven Soderbergh and Brad Pitt are in talks to direct and star in "Moneyball," an adaptation of Michael Lewis' excellent, excellent book "Moneyball: The Art of Winning an Unfair Game," a glorious story of stats-obsessed people in a stats-obsessed game.
-
"If I could leave you with a single thought from this post it would be this — your web designer does not work with you, he works for you."
-
Acronym acronyms — it's an acropalypse!
-
My first punt at an iPhone web app. I used Dashcode to write it — it's JavaScript, CSS and Canvas, with a Guardian-provided XML file of the tax data on the back end.
iPhone/Safari/Firefox only…
-
A Watchmen related Twitter feed. Includes links to lots of viral videos, Flickr photos etc.
-
A quick web page I made to list people at THe Guardian who use Twitter. You can view each person's tweets in a separate tab, and it will cache each person's tweets as you download the,
It's all done client side using JavaScript, and it calls in some YUI stylesheets to handle the tabs consistently. Because all the calls ceom from JavaScript on the client, the API rate-limit is done per user-browser, allowing me to deploy it without burning through the rate limit from a single server IP.
While I was writing this I did actually ask Twitter to whitelist my server IP, and they got back to me the next day to say this had been done, so next time I'll be able to do something useful using PHP and not worry about the rate limit at all…
links for 2009-02-07
Saturday, February 7th, 2009-
Facebook provides API access to it's status updates
-
It's literally 'quite funny'. Especially Mr ET.
-
Apple's developer documentation is all over the place! If you're writing a Web App for iPhone, as opposed to a native App, this is the page you want.
-
Christain Heilmann comes up with the goods in yet another of his excellent accessibility focused, Yahoo! tools-using hacks
-
A look at how much effort our designers put in to ensure their work is displayed consistently across the Guardian site