Development at paulcarvill.com, the home of Paul Carvill on the web

link: paulcarvill on twitter

link: paulcarvill at flickr

paulcarvill.com

Hi, I'm Paul Carvill, I'm a web developer. I'm currently working as Technical Lead at LBi, Europe's largest digital agency.

I also like walking, cooking, Bollywood and rock 'n' roll.

Archive for the ‘Development’ Category

HTML Developer’s Guide for ADOBE AIR

Wednesday, July 21st, 2010

Adobe’s website is absolutely useless. Hey Adobe, why not put all your developer tools, documentation & code examples in fragmented, ambiguous and duplicate sections on your site? Oh wait, you already did… So much of the documentation is misnamed, out of date or just impossible to find.

So if you’re looking for information on developing for Adobe Air using non-Flash, non-ActionScript, non-Flex options then this appears to be the most definitive documentation that I was able to find on the Adobe site: HTML Developer’s Guide for ADOBE AIR. Hopefully it will save you some time.

Internet time

Tuesday, June 8th, 2010

Me and @agentdeal have both recently been noticing an increased interest in analogue clock-based time interfaces on the web. Pleasingly, the practice has been implemented across web development disciplines, proving that it’s not just loopy Flash developers who spend their time making utterly pointless widget-y gizmos — even Steve Jobs and his panting army of fanboys can have a go!

The trend seems to have started with bbc.co.uk’s lovingly crafted piece of old test card nostalgia, made using Canvas:

It continues in spectacular fashion with the new Times paywall site thetimes.co.uk, which has a heartbreakingly pathetic laurel-wreathed carriage clock, rendered in Flash, updating in real time, presumably sitting atop a digital mantelpiece as a retirement gift to the online news industry:

The newest kid on the block, and definitely the showiest, is the timezone-straddling multi-clock Flash display of Globe Trotter, purveyor of hand made luxury luggage:

What fascinates me about all this is that surely every single example here and elsewhere is completely redundant. Is there a device anywhere that doesn’t display the time and/or date somewhere in the interface? I’m pretty sure that anything running a web browser also has a clock or calendar in view a mere eyeball movement away. But that is what makes these internet clocks so great — each is a conceit made of whimsy and I love them.

How to handle a Python BadKeyError exception in Google AppEngine

Tuesday, June 1st, 2010

This drove me nuts an hour tonight. If you’re trying to get an entity key using something like:

yourKey = someString
candidate = Candidate( key=db.Key(yourKey) )

and you get a BadKeyError, it’s because the string you’re sending in isn’t the right length. Also, AppEngine doesn’t return None from the Key method but instead returns a BadKeyError exception. You can handle this by doing:

yourKey = someString
try:
   candidate = Candidate( key=db.Key(yourKey) )
except db.BadKeyError:
   # handle the exception here

Increasing memory limit for PHP in Rackspace Cloud Sites

Sunday, February 28th, 2010

If you install more than a few modules in your Drupal implementation the chances are that it’ll run out of memory and you’ll start seeing blank pages and odd behaviour. The fix for this is to increase the amount of memory allocated to PHP, which you can usually do in your php.ini file. But if your hosting is Rackspace Cloud Sites you don’t have access to the php.ini file. You must instead put your PHP settings in a .htaccess file in the root directory of your hosting space.

Here’s some example settings:

php_value memory_limit 96M
php_value upload_max_filesize 50M
php_value post_max_size 50M

Now, here’s the important bit: once you’ve made put those lines in your .htaccess file and FTP’d it to your webspace, the changes might not appear to have taken effect. I had to delete the original .htaccess file and upload a fresh one for my changes to be picked up. Hopefully this might help somebody else in the same situation.

N.B. Rackspace’s support had next to no idea of what I was talking about. Their Livechat support service was as useful as telephoning someone in a library and waiting while they went off to find the information. They obviously don;t support individual applications hosted by them, but I’d have expected a little more help.

Apache Continuum error: Provider message: No such provider: ’s’.

Tuesday, January 26th, 2010

If you’re trying to use Apache Continuum and get the error “Provider message: No such provider: ’s’.” then you probably haven’t provided the correct SCM Url format in your project information. Importantly, it needs to begin with something like “scm:svn” e.g.

scm:svn:https://example.com/svn/project/trunk

Tree command for Mac OS X

Saturday, January 16th, 2010

This outputs a structured view of files and folder of the directory you execute it in:

find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'

e.g.

.
|____.git
| |____branches
| |____config
| |____description
| |____HEAD
| |____hooks
| | |____applypatch-msg.sample
| | |____commit-msg.sample
| | |____post-commit.sample
| | |____post-receive.sample
| | |____post-update.sample
| | |____pre-applypatch.sample
| | |____pre-commit.sample
| | |____pre-rebase.sample
| | |____prepare-commit-msg.sample
| | |____update.sample
| |____index
| |____info
| | |____exclude
| |____objects
| | |____info
| | |____pack
| |____refs
| | |____heads
| | |____tags
|____CHANGELOG.txt
|____COPYRIGHT.txt
|____cron.php
|____includes
| |____actions.inc
| |____batch.inc
| |____bootstrap.inc
| |____cache-install.inc
| |____cache.inc
| |____common.inc
| |____database.inc
| |____database.mysql-common.inc
| |____database.mysql.inc
| |____database.mysqli.inc
| |____database.pgsql.inc
| |____file.inc
| |____form.inc
| |____image.gd.inc
| |____image.inc
| |____install.inc
| |____install.mysql.inc
| |____install.mysqli.inc
| |____install.pgsql.inc
...

ls -R does something similar, but structured considerably differently.

Get Git to ignore Python .pyc files

Monday, January 4th, 2010

This will tell Git to ignore the files you specify by listing them in a special file, which you will tell Git the location of. This will only work on your local installation of Git. If someone clones your Git repository and adds files of the type you are ignoring, those files will be noticed and tracked by Git unless they also tell Git to ignore those files. For more on ignoring files on a per-repository basis, look here.

In Terminal:

git config --global core.excludesfile path/to/excludesfile

Then create your excludes file:

vim path/to/excludesfile

and add the files to exclude, using wildcards if necessary:

*.pyc


Why front-end developers are so important to the future of businesses on the web

Thursday, September 24th, 2009

or How traditional businesses who have moved to the web regularly undervalue their front-end web developers, and are worse off because of that

Distinction
The roles of web developers and web designers have been around for over 15 years now, and the role of a client-side or front-end web developer started to mature into a distinct entity around 10 years ago, as the content-presentation-behaviour layer paradigm became embedded in people’s working methodologies (and, with the introduction of Google’s then-new search algorithm, when the need for cleanly structured, easily indexable pages became, for businesses, not just an aspiration but a necessity). Unfortunately the perception of the front-end developer’s role remains somewhat coloured by an early association in observers’ minds with the other, loosely related role of the web designer. The role of web designer is an extremely important and valid one, but it is very different to that of the web developer, and the lack of a clear distinction between the two, in some people’s perception, is unhelpful and does both roles an injustice.

Skill set
The web developer (sometimes also called a client-side developer, front-end developer, web architect or front-end engineer) has a huge skill set and a job description to match. They are often expected and required to excel in many disciplines, and have good working knowledge of many others. They exist at the point where art, design, interaction, programming and behavioural and performance analysis intersect. Given the time, support and ambition of a good business, being a web developer can be an extremely fulfilling job. However, the role of a web developer is often misunderstood within even the most progressive and well-meaning of businesses.

Perception
The danger can be that front-end developers, working in a user-focused area, are seen as performing a superficial function — applying a polish to the heavy lifting done by another developer, say, or that dread comment, “making things look nice”. Let’s be clear, making things look nice is the sole responsibility of the designer. When front-end developers spend much of their time deploying underlying data received from a backend database into their views, or pages, they might mistakenly be thought of as merely translators or interpreters, transferring a graphical image — the Photoshop-ed design — into markup and style rules, purveyors of what is sometimes almost mockingly referred to as a ‘black art’ of making pixels lay out correctly onscreen. While this perception is perhaps unfortunate, it is understandable. It is a particular problem where a development workflow is — some might say artificially — segregated into database infrastructure/domain modeling/server side workflows/front-end workflows. In smaller organisations a front-end developer has the opportunity, if she wishes, to input into any of these areas. In larger organisations, the increased granularity of functional areas means those opportunities are greatly reduced, and as you can see from the segregation model above, the front-end development work comes at the end of a long chain of events and decisions which essentially shape and restrict the front-end developer’s choices.

Frustration
In such cases the development workflow is one-way, negates the developer’s architectural, organisational and behavioural skills and occurs late in the development process. This chronology minimizes the opportunity for the front-end developer to have effective input into, and feedback from, the interaction design they are now expected to code. This is a sad state of affairs and undoubtedly leads to frustration, feelings of being undervalued or ignored, and an extreme cases disenfranchisement and resignation, either in the figurative or practical sense. A good business will understand how highly-nuanced user behaviour is, and value skilled interpretation and shaping of that behaviour in the interests of improving their digital offering.

Value
The modern web developer has huge amounts of value to offer a business. Indeed the type of professional you often find in this role encapsulates the very best the web has to offer:

  • up-to-date knowledge of available and emerging technologies
  • extensive experience of implementing de facto web standards and programming patterns
  • database configuration and data manipulation
  • implementation across multiple platforms and legacy software applications
  • provisioning for mobile devices
  • data aggregation
  • graphics sourcing and creation
  • search engine optimisation (SEO)
  • a thorough understanding of the aesthetics and parameters of designing for the web

Further, the best web developers have wealth of knowledge and understanding around interaction design, user needs, hierarchies of data, navigation systems, user journeys, wireframing, design brief interpretation, focus group and usability testing and the art of a finely polished product. Largely gone are the days of HTML-monkeys, spending days on end converting Photoshop comps to pixel-perfect layouts. A web developer’s role is broad: from developing in what Yahoo!’s Nate Koechley calls ‘the world’s most hostile development environment’ — the browser — and ensuring cross-platform and cross-browser consistency, to working with art directors and designers and remaining true to their vision, to considerations and implementations of accessibility, usability and the overall user experience. A web developer is responsible for everything that sits on the client side of the web stack — the content, presentation and behaviour layers. Few other roles touch so many other key aspects of a business as does a web developer’s. Good businesses realise what an asset they have in their front-end web development team, and welcome their input into the product development process. Even better businesses have a User Experience team which encapsulates all those values, skills and judgements necessary to make great websites. Members of those teams are part of a feedback loop that results in great products, not just acceptable implementations of the first good idea that came up.

Slow
Large businesses and organisations move slowly. They may find it hard to understand they have developers whose skills and interests cross the boundaries their job descriptions impose on them. In addition, large businesses like to modularise their development teams into clearly segmented areas for planning and accountability. End-to-end developers don’t really fit this business model. Web development, certainly rapid prototyping at least, is moving away from monolithic relational database installs and towards schema-free, fluid data repositories like CouchDB and MongoDB. Many other layers of the application stack are now capable of being managed by a web developer. Most developers of this ilk, who are able to own the whole process from from domain and model definition, through to server infrastructure and on to a useful and appealing user experience, are running their own consultancies or are employed by the more enlightened web properties. Some examples of this type of person are Jeff Croft, Dan Rubin, John Resig, Jeffrey Zeldman, among many others. Functionality, data storage and interaction are increasingly moving to the client side (HTML5, Gears, RIAs, iPhone and Android web apps). The web stack sits on top of any technology, making the web developer one of the most versatile members of any business, let alone the technology department.

Undervalued
Unfortunately my experience has been that most large businesses massively undervalue their web developers, employing them in narrowly defined roles as the guys who make the site look nice, or fix the Javascript bugs that make the page break in IE. Larger business and the public sector have made moves towards working seriously on accessibility and usability, but the thinking behind such strategies remains superficial, those conceptual areas misunderstood. Too many people think of them in terms of awards and rating levels, not in ongoing process of improvement. Most of this work is also likely to be outsourced to a specialist third-party.

Career progression
One further, worrying, complication is the lack of clear career progression for front-end developers. Once you’ve spent a year or ten working in every nook and cranny of every browser out there until you can code progressively enhanced web pages blindfolded, what next? Economics and management structures mean there’s only so many architect or senior engineer roles to go round. The other option is to specialize in something and move forward from there. Current trends would seem to be leaning towards a big need for performance specialists in the next 5 years, as the client side moves ever further towards accommodating distributed, complex web applications. Page views will also continue their inexorable rise, placing stress and demand on infrastructure, databases and hardware, and thus even greater stress on a fast, responsive user experience.

Know your value
What does this mean if you’re a front-end or client-side web developer? Know your value. What are your skills? Are you a developer, an engineer, a User Experience architect or an Interaction Designer? Advertise your value. Shout about it. Don’t be knowingly undermined or ignored. Create a User Experience team within your business. If they’ve already got one, join it! Your job is incredibly important, and your present employer needs to realise that, as they stand to benefit.

Updated, 29th September 2009: just fixed a couple of typos

Wallpaper* Open House map is a prime example of awful Flash interface design

Saturday, September 19th, 2009

Further to my previous post regarding the Adobe-Omniture deal, and how it might improve the consistently awful Flash user experience people regularly have to deal with, this morning I found a very good example of bad Flash interface design to illustrate my point with.

The weekend of 19th and 20th September is London Open House weekend, when the public gets wide and varied access to the architectural pleasures of the City. Unfortunately, while Open House do publish a printed programme, the only way to discover locations of Open House participants online is through a fairly complex and ugly search form at http://www.londonopenhouse.org/public/london/find/. This method is not an intuitive way to find Open House locations, and does not encourage discovery or serendipity.

However, there is a map at Wallpaper magazine’s website. While this could have been a useful alternative to the official listings, it is instead problematic and discouraging. The Wallpaper site features a Google map, overlaid with a Flash map. There are several problems:

  • The Wallpaper Flash map features only a selection of the full listings. They call this the Wallpaper Edit. For the full listing you must still consult the printed programme or use the official search form.
  • The Wallpaper Flash map is extremely simplified, and lacks context, scale and travel information, as well as several other navigation aids.
  • The Wallpaper Flash map uses an entirely new and different navigation method. There is no drag control, and no zoom. Instead, the user needs to click on buttons near the top of the map which signify arbitrary areas of London. There is no option to show partial areas or the whole map at once.
  • The Wallpaper Flash map uses an entirely different scale to the accompanying standard image-based Google map. Changes in the orientation of one map are not reflected in the other, making switching between the two extremely disorienting.
  • The Wallpaper Flash map uses bespoke markers and a bespoke popup system, which opens a popup at the top left of the map, completely out of context of the click event. The Google map operates in an expected way, much as other web standards-based online maps do, by opening the popup at the location of the click event, where the user is focused.

The frustrating thing about Wallpaper using a custom Flash interface is that the Google map view already provides all the functionality the user needs, in an established, expected way, and in many ways in offers a superior user experience. Control of the Google map is more granular and direct, and the user benefits from the extra context and transport directions functionality. In addition, in case of the popup, the Google map provides more information, in the form of opening times. Even simple things, like the detail text being copyable, make the Google map a far more suitable implementation.

The Wallpaper Open House Flash map interface is a perfect example of the continued redundancy of Flash interface design, it’s ability to encourage design abuse, the lack of understanding of user behavioural patterns and the skewed importance of brand over usability.

Technology can, and will, save the newspaper publishing industry

Tuesday, August 25th, 2009

Nik,

This started out as a comment on Nik Silver’s post but quickly turned into a post of its own.

I won’t get into the details of the extent to which the newspaper industry is dying, or why, because there are plainly more than enough commentators doing that already, many of them ill-informed and/or self-important. It’s pretty easy to stand on the sidelines of an industry experiencing a decline, in circulation though not necessarily in power or influence, and shout out well-meaning advice, much as one might stand on a river bank and shout “try boogie boarding” to a swimmer experiencing some trouble in the choppy water. But for all their chutzpah, some of these commentator’s suggestions do contain elements of truth which the newspaper industry would do well to acknowledge, and indeed sections of which have already done so.

Act like a startup

The point which irritates you so, Nik, understandably, the instruction to ‘act more like a startup’, is a naive one, and I’m not altogether sure the people shouting it from the river bank really understand what they themselves mean. But in its naivety and potential for misunderstanding there are some interesting concepts. ‘Startup’ is a useful shorthand amongst the never knowingly buzzword-averse self-congratulatory internet community for the kind of dynamic, vibrant, go-getting bunches of technically savvy kids that bubble up on a regular basis from sun-kissed California or, at a stretch, from the environs of a grey roundabout in London, to invent a cure-all solution to everyone’s online needs that will change your life, alter your children’s futures and also enable your fridge to tell you when you’re out of milk, and whose businesses seemingly run on smiles and laughter instead of cold hard cash.

People in other industries don’t run startups. They run ’small businesses’ — decidedly less glamourous — or, gasp, they are self-employed — about as far from glamour as it’s possible to get (all that paperwork…) But most people have never worked for a startup, and so don’t understand how they operate. Most startups do actually have investors, whether they are parents, friends, extended family or even plain old bank loans. Further, any startup you’re ever likely to hear about through TechCrunch or similar has proper, big investment from other businesses, and a smaller portion still have insulation from market forces and major investment from venture capitalists, themselves usually branches of multinational investment banks. So, how to reconcile these two views of startup culture? The answer is that startup culture is a concept based on a perception of the startup ecosystem as a whole. That’s where the idealism bordering on the fantasy originates. But while the distinction is important, it’s the perception that we’re interested in, and that could be helpful to the newspaper industry.

Culture and DNA

Simon’s point, that startups and pre-web established businesses have vastly different origins and thus business models — different corporate DNA, as he says — is a good one, but even he would have to admit that that is a weakness of the newspaper industry, which will need to adapt to survive.

My point, which I’ll illustrate here, is that when people tell newspapers, or any other industry, to act more like startups, the are invoking the visible perception of the startup culture as a whole, not any single element of it. And of that whole, people see only the successes. Of course, there is one benefit of this view — your sample is self-selecting — we only need to look at those startups which became successful and ‘made it’ (got bought, IPO’d) because they must have been doing something right. But if ‘traditional’ businesses can harness just a little of the spirit of that culture, the — yes — idealism of it, then they’ll be well on their way to changing themselves for the better.

Readiness to fail

To outsiders, startup culture shows a willingness to experiment, and a readiness to fail. Never mind that startups themselves are atomised to a massive extent — there are a dozen startups devoted just to monetizing the upload of pictures to Twitter, itself still, ostensibly, a startup — to an observer the startup ecosystem is willing to try any possible avenue of revenue, no matter how niche, and if it doesn’t work to try another one. Low overheads and low headcounts means it’s relatively pain- and friction-free for a startup to close down and it’s proprietor to try again another day. Many entrepeneurs are famously serial startup start-uppers. Newspapers, indeed most big businesses, for good or bad, can’t afford to try their hand at every passing opportunity that comes along. They have overheads, high staff levels, investors and strong unions to consider. But those same factors also mean they have the infrastructure and the backing to take well-considered risks, and they should take risks more often. One hindrance to a newspaper is the line of editorial policy that runs through everything they do. Newspapers, to some people’s fury, sometimes represent more than just business interests. They represent a political ideology, a social responsibility. They also represent decades or centuries old brand names. But the industry is surely mature enough that a newspaper can diversify its voice enough to allow some of its ventures to fail admirably? What is a brand name if it’s not strong enough to absorb an under-performing product.

Relentless innovation

To outsiders, startup culture looks relentlessly innovative. And it is. They’re often solving problems you never even knew you had. Usually small, driven by a strong independent spirit and urge to be the first to market, they are unusually open, agile and responsive to customer requests — having ’strategic agility’, in Simon’s words. They literally cannot afford to lost their market niche due to bad software or bad will. To this end, one of the most important points is that startups are not afraid to be in beta. Having your software released in beta status has become such a fetish it’s almost a parody. But the meaning is plain to see — the software is a work in progress, and will progress based on customer feedback. To come out of beta signifies an end to progress. I’m sure we’ll grow out of this silly fad, but for now it’s a useful shorthand that your software vendor is listening to you.

Technology can save you

The last point I’ll make, probably the most important one, is that startup culture — the culture everyone in the media is referring to when they talk about startup culture, not small business culture and your local council’s small business forum — startup culture is technology-led. That is, technology is the driving force behind the products, the innovation, the business itself. Technology is the fabric of the new economy, it pushes at what’s possible, and an industry has sprung up to exploit that. Three of the biggest failing industries right now are the music industry, the film industry and newspaper publishing. Three content-led industries right there. Run for years, decades, by content people. Content people and lawyers. While content producers excel at producing what they produce, and lawyers will always be necessary, neither of these groups is in any way qualified to predict where technology could take them, or how it threatens them. The music industry in particular buried it’s head in the sand and hoped technology would go away. Hollywood got a clue, a little too late after spending most of it’s technology energy on DVD copy-protection, ignorant to the fact that for every developer writing encryption software there were 10 more able to write decryption software. The music industry is pulling out of its nosedive. So how did these two behemoths manage it? Technology. The film industry, led by technology, is deploying digital cinema distribution, special 3d camera’s and projectors, and, led by sony, a technology company, Blu-Ray discs for home viewing. The music industry has been pulled out of its hole by, among others, Apple, and I’m sure Amazon and Nokia are helping out too. That leaves newspapers. While certain players are making headway based on technology platforms which were designed and built this century, many, many more are struggling to even comprehend how technology can not only help them, but save them.

I have the luxury of speaking from the trenches, working daily on the nuts and bolts of news on the web in the software development of a national newspaper, rather than on the board of that same newspaper having to make massive strategic decisions about the future of the company. But, if I had one piece of advice for anyone working at a newspaper and wondering where the future was going to happen, or how it might help them, or how they might help themselves, I’d point them in the direction of their technology department, if they have one. Go grab a software developer and ask them what they could build for you. If it’s one the web, it starts here, in ideas and code…