The idea of the ‘architect’ of this system was that any interaction with databases or storage of data; ie the models; would be shared in the central library.  This would mean that changes such as the one I worked on with is_deleted = 1 would be simple and made in one place, and one place only.  A true implementation of the ‘Don’t Repeat Yourself (DRY)’ principle.

The nature of this platform is that it is exactly that. A platform.  This typically means that no two implementations of the platform are identical, so to speak.  The platform is built upon and ‘extended’ to do what the individual customers want from their online shop.  This sometimes means that custom product ‘getter’ blocks of code are written.  Often, the ‘quick and easy’ approach had been taken with these , and they had simply been added as standalone functions in the individual project’s library, called things such as getAllProductsFlaggedAsSpecial()  This obviously meant that when I implemented my is_deleted changes in the central library, and wasn’t aware of the custom nature of these custom product selectors, the sites that used those custom product selectors continued to display deleted products.  Big oops!

This now means that I will need to go through the individual implementations of the platform, search for any custom product SQL that is being built, and add the AND p.is_deleted = 0 in to them.

This whole problem got my thinking about how I would approach this issue (The need to build custom product getter functions in individual implementations of the platform), and here’s how I think I’d do it.

I think I would ensure that all custom product getter functions were in fact methods in an extension of the main Model class that then called a buildWhereClause method that added any global product SQL (eg AND is_deleted = 0) in, like this:


class Model_Product extends Base_Model {
...
function buildWhereClause($sql) {
return 'WHERE p.is_deleted = 0 AND '.$sql;
}
...
}



class MyShopNameModel_Product extends Model_Product {
function getProductsFlaggedAsSpecial() {
...
$sql = "SELECT * FROM product ".$this->buildWhereClause('is_special = 1');
...
}
}

Essentially, what I learnt here was that the developer to begin with that wrote these ‘quick functions’ saved his or herself half an hour or so, but cost me (2-3 years later) a fair amount of time in debugging.  I think this is evidence that coding things the quick way and implementing a fast solution is often a false economy, and someone later in the life of the software will have to spend the time saved (and possibly more) to work around the disadvantages of the quick approach.

• HTTP requests are expensive in terms of time.  Reducing the number speeds page load time.
• Browsers can – and do – cache effectively.  Inline CSS and Javascript can’t be cached for later page requests.
• Images can be optimised on the web without any major loss in visual quality.

Javascript Files

Minify Them!

Firstly, remember that if you are including libraries such as jQuery, there are a few ways in which you can do so.  They offer a “minified” or “compressed” version, which is at least half the size in KB of the uncompressed version.  Given that you are extremely unlikely to need to ever modify the jQuery core, I don’t see any argument for including the full version of the library, and would choose to include the smaller, minified version every time.

If you have your own javascript files that are particularly hefty, you can minify those for production sites too, by using tools such as JSMin.  The only problem with doing this is you tend to need to maintain two versions of the file.  The minification process tends to be irreversible so you will need to keep a “development” copy of the javascript, just in case you need to make changes or additions in the future.

Use a Content Delivery Network for your javascript libraries!

If you want to go one step further, you can include libraries such as jQuery from “Content Delivery Networks”.  Here is what wikipedia defines a CDN as:

A content delivery network or content distribution network (CDN) is a system of computers containing copies of data, placed at various points in a network so as to maximize bandwidth for access to the data from clients throughout the network

Put simply, CDNs are probably much more capable of delivering the javascript files to the web browser accessing your page quickly and reliably than your own server.  Additionally, remember that this means that the CDN is taking some of your bandwidth costs away – at least 50kb per new visitor.  An additional strange secondary advantage is that it is possible that the user has previously visited a page including the javascript library from the same CDN earlier in their internet browsing session, so it will already be cached in their machine, and will not need to be loaded for a second time.

If you are looking for a quick link to the Google Javascript CDN,  you can access it here:

http://code.google.com/apis/ajaxlibs/

Include them only when they are needed!

Imagine that you have a “google map” page on your web site that integrates with the Google Maps API.  You have a fairly chunky 15kb Javascript file that handles your personal Google Maps integration, as well as including Google’s own API file for the maps.  Why would you want this javascript on every page of the site?  A lot of web developers and designers are guilty of putting the javascript in every single page of the site, which means that the request and corresponding bandwidth associated with the maps integration would be used even if the visitor never viewed that area of the site.

Images

This section is going to be quite short, as it isn’t really my department, but my main understanding is:

• Adobe Photoshop has a save for web feature. Use it.
• Application specific META data is not needed in images on the web (I understand that Adobe Photoshop again can be a devil for this). Remove it if you can!
• If you have “hover” states on buttons or navigation items, use CSS Sprites rather than two separate images.  The reasons why are explained in the link supplied, but mainly revolve around making the hover work immediately, and reducing the number of HTTP requests.

CSS

There are two situations here, really.  The majority of the time, most of the CSS will be used on all of the pages of the site, so it makes sense to make this one external cacheable file.  It is important that the CSS isn’t included in <style type=”text/css”></style> tags in the document itself, as this information is not cacheable and will be individually reloaded on the following page.

A lot of sites will include two or three CSS files. One for the site itself, and one for a plugin such as “lightbox” or “slimbox” or something similar.  If lightbox/slimbox is used on most pages, why not combine the CSS files in to one slightly larger one?  This will reduce the number of HTTP requests and put all of the CSS in to a single cached file.

There is an argument, also, that if there is a particularly complicated section of the web site that takes up 10kb of a CSS file all by itself, then that 10kb of the CSS should be split out in to a separate css file and included only in the complicated section, where it is needed.  This kind of optimisation will need to be assessed on a per-case basis!

Finally, remember that css itself can be written efficiently.

Compare these two blocks of code, which do the same thing, and you will see what I mean!


margin-top: 10px;
margin-bottom: 15px;
margin-right: 5px;
margin-left: 2px;



margin: 10px 5px 15px 2px;

In conclusion, the steps above often take less than an hour to implement on a well built web site, and can save a couple of seconds per page load, and a fair amount of bandwidth costs.  With google’s recent introduction of a small focus on page load time in terms of a web site’s ranking, these kind of improvements have more benefits than just making the site appear faster, and are worth paying some attention!

I have worked for both technical and non technical people, and can say that it’s not only whether or not the app is finished that is judged based on the quality of the UI.  A more important factor that is judged is the productivity of the development team.  If you are a developer (which you might be if you’re reading this blog), you will know that writing models and controllers is particularly time consuming with little “real time” reward – that is,  people who aren’t programmers can’t see and don’t understand the work that you have put in.  Sometimes, this can lead to these people thinking that the programmers are doing nothing but twiddle their thumbs and get excited about charsets, or whatever it is programmers do.

Unfortunately I don’t think there is a lot that can be done about this, other than perhaps using these two useful analogies.

• Writing an application is like building a house. The expensive and time consuming part is the groundwork to prepare the site, the digging of the trenches for the foundations, and the laying of those foundations.  During this time the client thinks that you’ve just made a big mess and spent lots of their money, and they haven’t got anything resembling a house to show to anybody.  Additionally, if the foundations aren’t laid correctly, the reset of the building will be more likely to fail.
• An application is like a car.  It doesn’t matter how much time the engineers spend perfecting the handling and power delivery of the vehicle.  If it doesn’t look nice, the majority of people won’t like it, and even more people would decline the opportunity to buy it.  The engineering team needs to be backed up by a telented designer to make a successful product.

The judgement of quality or completeness of an application makes it difficult as a developer to put together any kind of portfolio.  You want your portfolio to *look* good – in full knowledge that people will judge the sites you have built on look and feel over functionality – despite the fact that you are not applying for a designer’s job at all.  But quite often the projects that look the prettiest are the most mundane and simple in terms of technology behind the scenes.  In addition to this, it’s rare (but not impossible) that you find an interviewer who is genuinely interested or excited by the way that you have achieved things technically, so talking through how you fully integrated your web application with Microsoft Sharepoint won’t interest them.  After all, all those months of development and there’s nothing fancy to see, only an “Update Website from Sharepoint” button in the web app’s control panel.  Really, it took 3 months to add that button?  I have a cousin in London who can do that kind of thing in an hour or two.

The Concept

The concept behind this is simple.  Every developer can be responsible for the code in any project.  As mentioned above, this means that bug fixes can be made by any developer, and new features can be developed by any developer.  Here’s wikipedia’s current take on the matter:

Collective code ownership means that everyone is responsible for all the code; this, in turn, means that everybody is allowed to change any part of the code.

The Advantages

In my opinion this introduces the following advantages.

• If a member of your development team is immersed in an important project and needs to be allowed to “get in the zone” and work interruption free, then if code they have written is deemed to be broken, any other member of the development team can fix the problem.
• This increases exposure of application code.  The more developers that have seen code and got their head around the way it works, the better!  When the developer that wrote the code leaves, it would be helpful to have other people who have at least seen the code and knows that it exists!
• Fresh pairs of eyes can foresee other problems with the code.  Imagine constructing a delete query using a function called constructInClause, which is passed an array.  This will return an SQL IN() clause statement.  “Hang on a minute, what if the function constructInClause returns an empty string? Won’t that delete everything from the database?” – better to spot this before rather than after the bug is exposed.

The Disadvantages

With every new approach comes disadvantages alongside the advantages.  Unfortunately, I do see a couple with this one:

• Allowing other developers to fix bugs and modify code can mean that they will not necessarily bear in mind the full knock on effects of a change.  Dependencies on code working a certain way are commonplace – even if not ideal – in web based applications.  (Note: Unit Testing could probably avoid this situation!)
• There is a danger of one person in the office becoming the “bug fix person”.  Not many developers enjoy fixing bugs all day, so in my view it would be important to try to make sure that responsibility for fixing bugs is shared equally between the development team.

Will I be encouraging the company I work for to encourage collective code ownership?

Probably, yes!  It seems like a very sensible idea, especially considering that at times interruption to fix bugs is really quite infuriating!  If the bug fixing could be assigned to the “most interruptable” developer at any one time, I’m sure that productivity and developer mood would both be improved.

1. Make sure you understand the problem in full.

Generally, when we are presented with a problem by either a client, or a project manager, the problem has been simplified somewhat.  A 15 minute phone conversation or meeting is condensed in to a three sentence paragraph asking you the best way around a problem.  In my eyes, there are a couple of approaches to avoid this.

1. Be involved in the meeting, or the phone conversation.  This is not always possible – not every developer wants to be interrupted every 30 minutes by client questions, and not every developer is necessarily comfortable talking to a client.  But listening in on (and not participating in) a conversation could always be an option!  This means that you get to hear everything that the client has said about the problem.  It’s to be expected that the person relaying the message will miss out minor (but not necessarily insignificant) details.  After all, repeating every phone conversation in full could get a bit tiresome, you’d imagine!
2. Know to ask the right questions. A client asks to make their delivery fee increase with every product added.  Let’s say, increase the delivery fee by 30% for every additional product that needs delivering.  This is simple enough, until you consider that the delivery fee could easily escalate way out of control.  A £10 standard delivery fee could soon escalate to around £50 if someone bought a large amount of products.  This could discourage a purchase.  So should a limit be introduced for the maximum delivery fee?  Remember, the client probably hasn’t thought around this issue.  Asking the question and getting an answer before work starts could save you from having to rewrite a chunk of code.

2. If dealing with unfamiliar technology… allow some research time!

If you’ve been asked to implement an unfamiliar technology, take 10 or 15 minutes to research how it all works.  15 minutes spent now could stop you from estimating that something will take 5 hours when it might take 20, or from estimating 20 hours when it might take 5!   Additionally, when estimating how long the work will take, remember that you will probably spend a fair chunk of time reading documentation and not actually writing code.  In many places this would be considered billable time – and it should certainly be taken in to account when working out how soon you could deliver the work.

3.  Remember to allow time for testing and bug fixing.

Not everyone does this, but in my opinion it is essential.  Allow some time for proper testing and the time it’ll take to fix bugs (this can only be estimated) as well as the time taken to write the code in the first place.

4. If dealing with third party services, you can’t accurately promise a deadline.

A lot of the work I do involves integration with third party services – payment gateway services, SMS services, etc.  These all involve an account application and signup process, and sometimes this process depends on the client completing a form, or submitting payment for the service.   I have found that it is almost impossible to accurately estimate a “go live” date when an aspect of your application depends on a third party service being ready for use.  If you do provide an estimated go live date, it could be worth putting in a note that it depends on the client’s service X application being complete by date Y.

5. If you feel out of your depth, don’t be afraid to say!

This is a very important point.  If you are not confident about working with and fully understanding the technology that you will be using, tell your manager or your client! It’s better to say at this stage than to blindly assume that you will be able to work something out during the work.  They will be a darn sight happier this way around than if you go out of your depth and then the project is late or massively over budget.  Saying now typically will have one of three results:

1. Your company will turn the work away to avoid a problem (or, your client will go elsewhere).
2. Measures will be put in place to avoid the issue. Billing by the hour, or using another developer, for instance.
3. The project will be quoted anyway at an estimated rate, and your company will take a risk. In this case, you at least said something at the start – the risk taken was by someone above you.

These are only very basic steps that can be taken to avoid a problem.  But one thing is for certain.  It is typically better to under promise and over deliver rather than the other way around.  You just have to be careful that you don’t fall in to the habit of *massively* over-quoting projects, because this can have a detrimental effect in terms of reducing the percentage of accepted quotations.

My reasons for this decision are as follows:

1. Database collation.  Searching for strings LIKE '%Ed, Ed & Eddie%' starts to get complicated if you have converted & to &amp; – you have to remember to convert entities in your search string as well.  This could also potentially mess up the effect ORDER BY clauses in the SQL queries you write have on the dataset.
2. Data Integrity.  If in the future I decide that I all HTML, or just specific previously blacklisted HTML tags to be allowed, the data is all still complete and certain aspects of the data have not been removed.
3. Application Consistency.  I know that if I am ever displaying data selected from the database it will need to be ‘sanitised’ before being displayed to the user either in repopulated ‘edit’ forms or in a normal ‘view’ type screen.

I also did a little bit of research in the cakephp manual (as we will be using cakephp for this next project) to see what cakephp creators thought about this and it seems they reached the same conclusion (quoted from this page):

For sanitization against XSS its generally better to save raw HTML in database without modification and sanitize at the time of output/display.

They don’t give any specific reasons but I imagine that their thought process was similar to mine.

Starting with the basics, organisation in projects can be as simple as ensuring that you store all information supplied by the client with the project, in a folder called something like ‘assets’.   Typically I would split them by asset type – images for web, images for print, documents for web download, internal documents, other – and perhaps include a ‘supply date’ in the filename of each.

Next, write stuff down!  If a particular part of the project depends on an external service being available, write this down in a document that stays with the project wherever it goes.  Any useful information that is in your head the day you write the code for the project should probably be written down.  You more than likely won’t remember in 2 years time when something goes wrong!  All of this information could simply be stored in a README file.

Good organisation doesn’t boil down to simply putting things in to folders, or keeping notes.  Of course, these actions are a part of the organisation, but don’t cover everything.  More importantly – especially in the IT industry – consistency can be the best time saving characteristic you can introduce to your work.  Being consistent is almost synonymous with being organised when it comes to your project management…

I save time (and frustration!) by:

• Naming projects consistently.  My project name tends to match the main part of the domain name on which the web site or project will be hosted.  If I’ve a domain called edwardyarnold.co.uk, the project in my subversion repository will be called edwardyarnold.  This makes it a lot quicker to put my hand on the correct project if a change needs to be made.
• Naming the associated MySQL databases consistently.  For the above project, the CMS database would be called edwardyarnold_cms.  If the site were an ecommerce site, it would be called edwardyarnold_ecom.  This makes finding the relevant database in a phpmyadmin screen much quicker than if the databases weren’t named consistently.
• Ensuring that if a project is replaced with an entirely new version, it is archived with a date.  If I replaced the edwardyarnold.co.uk site with a completely new build, I’d move the current project to edwardyarnold_2010_archive and create a new one to replace the old.  This is a better approach to dating the new version of the site – the new version of the site would still exist in 2011 but would appear to be out of date if dated 2010.  Annoying to anyone who didn’t know that that was the ‘latest’ version of the site.
• Naming administration area directories according to a set naming pattern (I won’t disclose that here).  This means that anyone who I work with that knows the pattern can find the administration area without looking up the directory name.  (They will of course have to look up the username and password, if not saved in their browser).

These save a tiny amount of time individually, but collectively – and applied over multiple projects – the tiny amounts of time soon add up, and you can start to actually notice the saving.  Additionally, following these ‘rules’ means that new people joining your team can begin to understand and get a grip of the way you work quicker than if everything was done differently for each project.  In larger organisations this can reduce recruitment costs (remember, downtime whilst training your new staff member is a hidden recruitment cost…!) as well as saving your existing team time.

Until I worked for a couple of established Web Development companies, I never realised the importance of a ‘Checklist’ at the end of a project.  You know, simple things.  Does the site’s markup validate?  Do all images have alt tags?  Is php display_errors off?  But even after getting in the habit of ticking all of the items off this checklist for every site I build, there are still little mistakes that creep through – miscapitalised letters, incorrectly pluralised nouns… eg “There are 1 comments on this post” should be “There is 1 comment on this post”.   No checklist can possibly cover all of these possible errors (though the one I link to later in this post gives it a good go!).

The best way of discovering these problems is to ask a favour of (or even to employ) someone you know – preferably without a technical background – to go through the site and see if they can spot anything that needs changing or fixing.  You will find that they will probably produce a list twice as long as your original checklist!

The beauty of this approach is – as well as spotting spelling mistakes and other simple errors – you may also get some vital usability feedback, especially if you are with them when they are checking the site.  Do they seem frustrated? Did they use the site in the manner you would expect them to?

I was once fortunate enough to be behind a one-way mirror in a usability testing session and it was simply amazing.  We had a captcha image on a form (I know, I know.. don’t start) and the error message was misunderstood or found to be confusing by nearly every user.  The abandonment (and frustration) rate on the contact form was therefore extremely high.  This kind of problem would not normally be picked up in smaller projects where a usability testing session is not included (and you wrote the error message – so find it simple, right?), but if you run through the site with someone to whom the site is fresh, there’s a much higher chance that you will pick up the problem.  Remember that this issue wouldn’t be picked up by a checklist either!

So.. checking through a site with somebody non-technical (and who has never seen the site before) can save you a whole bunch of headaches in the future.  And remember – if you’re launching your web sites without a flight check list (check this one out from Box UK), that’s probably one of the first things you should consider implementing in your project schedule.  It will save you a lot more support time than it costs you.

Hardware (Performance)

Despite what some might tell you, web developers can not realistically get by on an old machine with Windows XP and 500kb of RAM.  We would actually like to get our work done “some time today”!  But, at the same time, we don’t need a kick-ass machine with 1gb of available Graphics Memory and 4gb of RAM.  The main things I like from a web development machine are quick application start up times and stability, both of which can be achieved with any modern operating system (I favour Windows 7 – sorry Apple fans) and a decent helping  (2gb?) of RAM.

Displays

I think that dual monitors are essential, and given the choice would never work on one monitor again.   I think that purchasing a second screen is a great investment.  Being able to have code on one screen and the application on the other (or email with instructions on one, code on the other.. the list is endless) speeds my work up so much more than I have ever been able to prove to anyone that has asked me to.  Try it – if you can – I don’t think you’ll ever go back to single monitor hell.

Web Browser

Despite it’s occasional large memory footprint, I don’t think you can beat Firefox if you’re a web developer – even if only for the incredibly useful web development add ons that are available.  I only really use two, though:

1. The Web Developer Toolbar – this has saved me more time than I could ever imagine finding why an image is fuzzy (damn CSS sizing) or why certain areas of the page appear the way they do.  It is just a huge time saver, one that in my opinion should be the first thing you install after Firefox itself.
2. Firebug – I find this most useful if involving myself with AJAX, it’s “net panel” is simply awesome.  I don’t even touch on half of this add on’s features, and still think it’s amazing!

Text Editor

I really, really like Activestate Komodo Edit, although I must admit that I haven’t tried anything else for at least 3 years now.  My favourite feature in this application is “Go To Definition”.  Right click a variable name, function name, or class name, click “Go To Definition” and you will be taken to either the variable assignment, function or class definition.  This has saved me loads of time!  The PHP syntax checking works well, and variable/function autocompletion is another big time saver.

FTP Client

When they made it paid-for a few years ago, I tried a few free-to-use alternatives and ended up back where I started, but with a fee to pay.  And in my view that fee is worth every penny.  I don’t think this FTP client can be beaten for ease of use or reliability -  SmartFTP.

Email Client

This will come as quite a shock to most people, but I tried using Thunderbird for a period of a year and got sick to death of it’s constant memory hogging.  Now, I use Microsoft Outlook 2007 for accessing my IMAP mail at work.  But, my favoured email client of them all is Google Mail.  I love how powerful and fast the search is, and fell in love with Labels as soon as they were introduced to me.  *All* of my personal email (And freelance email) goes through it.  If they made it paid-for, I’d pay for it immediately (within reason).

Scheduling Tool (Meetings, etc)

I started using Google Calendar in 2006, and haven’t looked back since.  The ability to share calendars to view, or view & edit, and the ease of synchronisation between this service and a whole load of mobile phones makes this an immediate recommendation for me.  Not forgetting of course that I can log in to google.com/calendar anywhere in the world and view or edit my schedule!  This is my favourite web-based app to date, and my favourite use of AJAX.

Before I go any further, I want to explain (or rather, quote someone else explaining) what indexing does.   A really useful source of information for me when I was investing this issue, was entitled “MySQL Cookbook”  – an 8 year old but incredibly useful book  that I have successfully counted on each time I have hit a brick wall with MySQL.

Typically, indexing a column that you query frequently helps SELECT statements run faster because the index allows MySQL to avoid full table scans. If you update this row more frequently than you SELECT or JOIN on it, you may be better off not indexing, as every time an UPDATE or INSERT query is ran, mysql must update the indexes for those columns.

Because a join can easily cause MySQL to process large numbers of row combinations, it’s a good idea to make sure that the columns you’re comparing are indexed. Otherwise, performance can drop off quickly as table sizes increase.

Example

You have a “product” and “product_image” table. The primary key in “product” is product_id. You have a column in “product_image” called “product_id”, which “links” the product image to the product that it belongs to.   product_id in both tables should be indexed!!

To give an example of the kind of speed increase this results in…

An application with this exact issue had a query which was taking on average, 8 to 10 seconds. After adding the index to the second table, that same query took only 0.02 seconds.  This is a quite frankly amazing difference, and all achieved just by indexing a single column.

Conclusion: As a general rule of thumb, any column in a database called (something)_id should be “indexed”. Exceptions will probably exist (in frequent-update and insert situations, for example), but if – as is often the case with web-apps – you SELECT more often than you UPDATE or INSERT, it’s a pretty concrete rule that you can stick to.