Category Archives: Web Development

Make Website faster

So you have a website and you want it to be faster. Perhaps you should of started off thinking that but in the real world that is often not how it happens.

Sorting out front end issues is often often messy or perhaps you have problems on the server side, some are hard to resolve. Sometimes you can be fighting against the framework package the website was built with. Other times you can quite quickly improve some things with a few tweaks to the server software.

Time spent waiting for the response is server side most after that is client side. Often the majority of the time is after your server has chucked out the html.

Reduce the size of the things to downloaded

– Reduce the size of files, compress those files over transmission.
– Use appropriate image formats.

Reduce the number of things to download

– Combine page assets to reduce number of page requests
– Setup a caching policy to remove the need to keep downloading the same assets.
– Put the whole page in cache, using a forward proxy or caching layer.
– Serve assets that don’t need cookies without them e.g. images

Do less

– Split up big monolithic pages into several e.g. paging
– Strip out parts of the page or site that are slow.

Do it asynchronously

– Does it need to happen immediately at page load or could it happen once the page has loaded.
– If you loaded the footer after a second or two would a user notice.

Backend

– Update your server software versions, databases, server side language and server itself.
– Look at protocols would httpd/2 be faster?
– If your using Apache would Nginx be faster for your circumstances.

Reduce the amount of time it takes to download assets.

– Use a content delivery network so files are nearer to the client.
– Split assets across multiple domains, so they can be downloaded in parallel depending on how your browser is setup.

Use faster DNS

– Once the DNS is requested its cached, you can say how long for but the initial first look requires your browser matching up the domain name to an address.

Do it out of sync

– Try prefetching the next page the user might go to or your other domain
– If they requested a report email it to them rather than make them wait whilst it is prepared.
– Only initially load images in the visible area of the screen and only load the rest as they become visible.

Simplify

– Use only those JavaScript libraries that are appropriate, do you need them all.
– Reduce HTML mark up complexity, do you need all those divs, or could they be made less, leaving yor browser less nodes to parse when building the page and reducing the page weight. Also can make your style sheet simpler if markup is more common across pages.

Assess JavaScript events

– If you sending a request on scroll do you need to do it continually or just at the end, is the server response timely or is it causing jaggedness.

Assess third party resources.

– Could you set a timeout on fetching third party resources, advertising, social widgets if they are not essential.

Check 404s

– check if there are any 404s triggered by requests they may not be as obvious as missing pictures, requests for odd favicons and robots.txt have a look in your logs.

Check server setup

– keep alive?
– etags, cache headers, created (telling the browser it already has the latest version so skip fetching same thing again)
– caches in database layers, or compilation in your server side code.

Translating a Magento shop

Been doing some translation for a shop built with Magento we wanted to try a French version of. For now it uses automated translation, in the future if it proves worthwhile we could move it to using a real translation.
I’m not a French speaker so its difficult to tell how good the machine translation is but we think its probably good enough to prove the feasibility of the scheme. Subtleties about French in France and French in Canada are a bit beyond us, but we are aiming at the French in France.

We made the French shop a Store View so it shares the same stock and pages as the main site but under a different domain name. Which seemed the simplest way.

Technically

We add the text block to a list of the product texts when the English text is altered. Then a PHP script goes through recently altered texts and gets the translation of that text in French. We also have a local translation cache somewhat inspired by, but much simpler than the Etsy scheme. This way we can save translating strings that have already been translated. In the shop I’m working on the short descriptions and full descriptions for products are actually the same so there no point translating them twice.

We are using Bing translate because their API seemed to make sense and they had a large free tier (2 million words) I could try stuff out on. It seems very solid technically it always seems to be there and hasn’t caused any issues so far. They also have a wide range of languages if your doing something more esoteric including Klingon ( pe’el maH roj ).

It was originally built on the Data Marketplace and now switched over to the newer Azure where its a bit hidden on portal.azure.com under Intelligence + Analytics > Cognitive Services API. If you have trouble creating an account / subscription I found they don’t like things they regard as prepaid cards, but don’t say that instead putting up a ‘problem with your account contact support’ message. Debit cards seem to work.

Magento makes translation not too technically demanding, basically you add alternative language versions of your CMS pages like contact us, about us, shipping, terms and conditions etc. Then you go through your various templates and check for any strings not in tags like this __(‘In Stock’); ?>
The strings are then looked up in the corresponding language files under app/etc/locale/fr_FR. If you haven’t got the language pack you can take a look on the Magento connect site and use one of those as a basis. Things that still come up in English need adding to the .xml files in lines
‘base language’, ‘translation’
style like
“Free”,”Gratuit”. There is lots of going through pages and checking for little English chunks.

You’ll also need to create/edit a switch template to allow users to switch languages and or currencies. In the way you want it to work and that is simple for the visitor to the site to use.

This was all in Magento 1.x;

Scumbags, Idiots and Charlatans

In the last few weeks experienced a lot of client abuse on the interweb.
Customers paying through the nose for web development and just being ignored.
Customers being abandoned.

I’ve been picking up the pieces for a couple of people and it makes me feel kind of sad its just that there isn’t any need for it. Ok things change and that is fine the web developer is not married to the customer and if you don’t have the skills in house to support them then that is fine but say that.

Not all customers are super technical and some need more hand holding than others many are actually frustrating.

But there isn’t any excuse really for treating people that badly. Building web stuff is a people business your coding stuff that will be used by humans in order to fulfil aims that other humans have. Whilst people are sometimes difficult and irritating and occasionally just downright maddening. Its developers job to help them, not impose stuff on them or extract money for nothing or exploit their lack of knowledge. If you can’t get on say that and leave them. Don’t just ignore them, you could even help them find someone new to sort things out.

Increasing font size in your browser

I was working on mobilizing a site recently and was asked about font sizes. Also was talking to someone who was ordering tablets for staff because everyone could read them because of screen size.
Traditionally you could put increase font size buttons on a site then store it in the session. I’ve done this in the past on public sector sites that were insistent on passing WCAG and WAI guidelines. While that is well intentioned I’m not sure its a great way of sorting out the problem or even passing the intention of the guidelines.
Users don’t want to be pressing an increase font size button on each site they visit that may or may not be there and is going to be in a different position on each site. It seems better for users to control font size in their browser either permanently or with the browser on an odd page. That way it is stored and more permanent.
Their eyesight isn’t going to change from website to website, barring laser eye surgery in your lunchtime.

That said its worth bearing in mind when designing sites everyone benefits from clear text and a site layout that stays readable even with the text resized according to browser preferences. Not only the middle aged or people with reduced vision but the tired eyed genius teenage programmer of stereo type.

The BBC offers good guides on text resizing if your trying to setup a browser for someone else.

Can I eliminate all errors?

Trying to move towards a zero tolerance for errors.

  • Even minor errors can obscure more major ones
  • Any error could be cancelling out another error – a kind of error ridden hell to avoid at all costs
  • Errors that seem insignificant now may be come more major issues in the future (when deprecated becomes removed for example)
  • Broken windows – lead to more broken windows.
  • Errors annoy me
  • Its hard to take any pride/pleasure in code that has even minor errors.
  • Every error is an opportunity to improve code
  • The cost of dealing with errors later is likely to be more
  • Errors can be an opportunity for a hacker to gain information about your system
  • Every error is an opportunity to learn – if you had understood it completely before that error would never have happened.
  • Errors cascade and create other errors

Its better to preempt problems before they occur in the wild. Users shouldn’t have to deal with errors or experience their side effects. Its better for the clients site and whats good for them is good for me (eventually). Its probably impossible to eliminate all but you can squash all the ones you know about. At least with every site reporting all errors however minor to a central source, any that occur over night can be sorted out by breakfast.

Having said that going through 404 logs does make you paranoid. There are bots out their trying to get you :). No really there are.

Stop Users using Stupid Passwords

Been some discussion recently on the WPUK mailing list after some recent mass attacks on WordPress Sites. Lots of good advice from people.

  • Do you need to to allow access from all IP’s
  • Do you need to be able to login all the time or can you say they never login a 2am in the morning so we can stop login at that time.
  • Keep up to date obviously
  • Changing default locations of login pages, wp-admin etc
  • using two factor authentication (yubikey mentioned)
  • captchas on login screen
  • blacklisting of visitors by IP address
  • renaming the default ‘admin’ user

All suggestions worth considering depending on your circumstances.
Someone mentioned about a user with a password of ‘password’, I wondered if there was a way of preventing users injuring themselves by using such a password. By default my current WordPress 3.5 allows a user to do pretty stupid stuff like create a user with a username of ‘e’ and a password of ‘password’.

Not added in list popular passwords that would pass as secure should do that. Could probably extend to exclude dictionary words not sure if its worth doing that.

Dave of interconnect.it mentioned a plugin called ‘better-wp-security’ which does quite a few other things as well, seems worth looking at, Dave had had it recommended to him. There are also a few similar plugins with more restricted scope that just stick to the password ‘force-strong-passwords’ and
‘enforce-strong-passwords’.
I’ve not tried these so I can’t recommend them, I bashed out my own because I was interested. Its probably not quite so client friendly as its called ‘Stop users using stupid passwords’, might have to rename that.

Seems something worth checking on especially as you create a new site. If the password isn’t towards the top of the list of ones a bot attacker might try, it will at least take them longer. Will also stop a human guessing it quite so easily as well.

When its all gone wrong

Sometimes your relationship with your developers can go wrong. What happens if they won’t do anything or won’t enable you to update your site?
Here are some terms its helpful to understand when sorting out the mess. Then some pointers on moving your email and moving your website.

DNS

DNS is the system that tells the internet where to look when someone types in your domain name.
Say your companies website is at www.example.com. When you type www.example.com into your web browser, your computer does a search and finds the IP address for that domain name. Its like sign posting system. This makes it quite crucial, if you can get control of your DNS you can point visitors to your site and emails to your anything@example.com to a new provider.

DNS records are a complicated subject but there are two main types

  • MX records – where your email is
  • A name records – where your website is

IP Address

The address of your server as seen by your computer so like 192.0.43.10 or 2001:500:88:200::10: pointed to by your DNS.

CMS

Content management system – the system that allows you to edit your site examples include WordPress, Joomla, Drupal, DotNetNuke, Umbraco, Django, Mura, etc, etc.

Moving your email

The things you will need to know to move your email to new provider are really just the list of email addresses. If you have the existing details (password, username, incoming server) that is ideal as you can use them to pull over any email from the old server.

Moving your website

This is a bit more variable, if you have a static site. It is just a bunch of files all of which are publicly available. So you can grab them or your friendly developer or hosts can.
If you have a dynamic site that has a database and some server side languages it is more difficult. For these the source code isn’t the same as what you can see so you have more difficulties. These sort of sites usually have pages that end with extensions other than .html or .htm for example (.php,.aspx,.cfm) or sometimes with no extension at all like example.com/my-monkey-submarine.

You can make a static copy of a dynamic site as you would for a static site but you do lose any CMS and the information that was in your database that wasn’t publicly viewable.

The vast majority of the time its not like this your existing developer will give you the raw files and database (that is the ideal case). Its your information, you paid for it and you should own it.

Your super idea

Everyone has brilliant ideas in the bath sometimes they even look good in the daylight. (see Archimedes)
You may be a bit hesitant to tell anyone about your idea. Here are some simple things you can do on your own to check out its viability.

  • Do some Google searches around your idea, is anyone else doing something similar?

    • Not to worry you don’t have to reinvent the wheel just have a different spin on it most businesses aren’t revolutionary more evolutionary.

  • Try it out on a few trusted friends, your spouse, partners
  • Have a look at domain names are they available for your idea, if people are using a similar name to what your thinking are they doing a similar thing to your idea.
  • Find someone who could be a potential client/customer ask them what they think.

Telling other people about your idea is a quick way of trying stuff out. You evolve your idea just by repeating it to people and gauging their reaction. Are they getting it or are they struggling to understand what your going on about.

You probably think everyone is going to run off with your precious new idea, to be honest they are more likely to just forget it. Also any enterprise is a lot more than just the idea, the execution is the majority of it. Did Microsoft become dominant because the were the best at original ideas or because they were good at execution?

Then you could try prototyping your idea, build a little prototype website only a cheap thing just a day or so in cost perhaps. Hold a test event. See if anybody is interested, really.

Maintenance

Picking up a website today to add in an advert. Found that no maintenance had been done on the site for five or six years, also there were no backups. As you would expect there were a few things needing fixing. Seems odd really you wouldn’t expect your car to last that long without servicing it, even if it was just cleaning it. Perhaps maintaining your site slightly more frequently would incur less costs than a big bang “oh my god its all fallen apart”.

Guess it depends how important your site is to you, if it went bang! and off for a few days due to a big problem would you be stuffed? Perhaps it would be an idea to have someone do the internet equivalent of checking its oil every year.

  • Check it works in browsers released since your site was created
  • Update any ready made systems your using, WordPress, plugins etc
  • Check you have backups of your files and or databases
  • Check it isn’t using deprecated functions

If this sounds self serving it genuinely isn’t most people will charge you a whole bunch for rebuilding an entire site that has got in a big mess. Compared to that small charges for simple maintenance are an economy.

Give your site some care.

Fuzzy testing

Been trying to work out what browser based testing is good for and not good for. Went to a talk to the other night at syncnorwich about testing at google scale. Seems they do use browser based testing but more as a top level of icing above a lot more tests of smaller units of functionality at code level.

It seems to make sense to test things as they are seen by the client, so I’ve been poking at Watir which allows for remote control of web browsers and therefore integration into automated tests. Apparently its jut one way of going about doing this and its a thin wrapper around selenium and web driver. I’m currently also interested in Ruby though so I went with watir (pronounced water apparently).

it seems browser based testing is good for a few use cases

  • Different browser
  • End to end tests
  • Random inputs

    • Pounding on a url with random urls

It seems like probably browser based testing is good for end to end overview testing. Making sure that even though all your unit tests work the web application works fine when all put together.

Daniel Wagner-Hall mentioned fuzzy testing to me which seems an interesting idea and something easy to automate but its not something I’d heard of before.
So I started on wikipedia

“Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.”

This seems like something that watir would be good for pounding on a page/site with random form inputs testing out back and forward browser actions, clicking random links etc. Cat walks across the keyboard style. The kind of thing a monkey would be good at one of the monkeys that aren’t busy typing the complete work of shakespeare. Never really understood why would a monkey would want to type shakespeare surely monkey fiction would be quite interesting?.

So anyway I’ve given my monkey something more interesting to do. You will need to have Watir installed/setup and I’m using Firefox but you could swap that out for your browser of choice.

web monkey script