Translating a Magento shop

Been doing some translation for a shop built with Magento we wanted to try a French version of. For now it uses automated translation, in the future if it proves worthwhile we could move it to using a real translation.
I’m not a French speaker so its difficult to tell how good the machine translation is but we think its probably good enough to prove the feasibility of the scheme. Subtleties about French in France and French in Canada are a bit beyond us, but we are aiming at the French in France.

We made the French shop a Store View so it shares the same stock and pages as the main site but under a different domain name. Which seemed the simplest way.

Technically

We add the text block to a list of the product texts when the English text is altered. Then a PHP script goes through recently altered texts and gets the translation of that text in French. We also have a local translation cache somewhat inspired by, but much simpler than the Etsy scheme. This way we can save translating strings that have already been translated. In the shop I’m working on the short descriptions and full descriptions for products are actually the same so there no point translating them twice.

We are using Bing translate because their API seemed to make sense and they had a large free tier (2 million words) I could try stuff out on. It seems very solid technically it always seems to be there and hasn’t caused any issues so far. They also have a wide range of languages if your doing something more esoteric including Klingon ( pe’el maH roj ).

It was originally built on the Data Marketplace and now switched over to the newer Azure where its a bit hidden on portal.azure.com under Intelligence + Analytics > Cognitive Services API. If you have trouble creating an account / subscription I found they don’t like things they regard as prepaid cards, but don’t say that instead putting up a ‘problem with your account contact support’ message. Debit cards seem to work.

Magento makes translation not too technically demanding, basically you add alternative language versions of your CMS pages like contact us, about us, shipping, terms and conditions etc. Then you go through your various templates and check for any strings not in tags like this __(‘In Stock’); ?>
The strings are then looked up in the corresponding language files under app/etc/locale/fr_FR. If you haven’t got the language pack you can take a look on the Magento connect site and use one of those as a basis. Things that still come up in English need adding to the .xml files in lines
‘base language’, ‘translation’
style like
“Free”,”Gratuit”. There is lots of going through pages and checking for little English chunks.

You’ll also need to create/edit a switch template to allow users to switch languages and or currencies. In the way you want it to work and that is simple for the visitor to the site to use.

This was all in Magento 1.x;

Onward, up grade

After upgrading several shops recently, I’ve been thinking about the upgrade process. Perhaps it should be central to the process of building an application. The ease of upgrading must help keep installed packages up to date, if its easier it seems much more likely to happen, if its automated even better.

To that end any tools that ease the upgrade process like wp-cli are great. I’m sure the ease of upgrade in WordPress must contribute mightily to their success in getting WordPress to its vast number of installs.

Even someone relatively motivated and informed when faced with an upgrade process that requires copying over a whole load of new files over old files feels deterred. I’m sure the effect is exaggerated in someone who doesn’t understand the error messages. Its also prone to error when you picking out different file paths /admin/theme/default/default/theme and /theme/default/default/theme. Even more frustrating if something that is supposed to have an upgrade process that then fails, as your trying to do the right thing and blows up in your face.

Its something I’m going to factor more strongly when choosing which packages to use for projects in the future.

Backing Up

Reviewing back up regime here so thought I would talk about it. Mostly this is done with the aim of making restoring sites easier after catastrophe. Sites are on a mixture of dedicated machines and shared hosting.
So you think about the things you would need if some thing ( think hard drive ) went bang poof in a cloud of smoke.

  • The database
  • Customer edited or uploaded files via a Content Management System CMS or LMS
  • Logs

So for example on a WordPress site might mysqldump the database and zip up the wp-content/uploads directory. Because I have the php files and theme files on a test server somewhere else like a laptop.

Storing

Once those backups are made they need usually to be kept of the machine, that you made them from otherwise your backups disappear at the same time as the original.
Its worth trying to use them I do this on my local copy of a site this reassures that the backups are working. It also makes sure the local site that I’m testing on has a reasonably representative and realistic set of data. Which is helpful as sites grow and things that were performing fine with Z amount of data might struggle with 10xZ.

Automation

Because backing things is quite dull it helps a lot if you can automate it. A backup job that fires every Y days is a great labour saver. Computers unlike people love this sort of thing. If you can automate moving the files away from the originating server its great too. Dropbox and Amazon S3 for example have code libraries that you can use as a basis for a file moving script.

Frequency

This is a bit of a how long is a piece of string question but you have to take into account how important something is and how frequently it changes. Some things I backup every night others only monthly. Some the worst that would happen is someone might lose a blog post other times they are taking orders and keeping records is more important.

How long to keep it

Not so sure about this I’ve got copies of some sets of files going back years. That is a bit pointless but perhaps that is right for you. Sometimes if the data over a certain age doesn’t change then you may only need to backup the newer files rather than keep backing up the same possibly large files month after month.

This is all aside from any backups the hosting company may keep and larger system wide machine images.

Making it part of development process

I do this for most of the sites I maintain and I’ve made writing backup scripts one of the things I do when I’m putting together a site which helps me remember it.

Beep Beep vehicle reversing

Bespoke Development

To make the case for bespoke development. There are a lot of benefits, your directly attacking the problem you have, directly and precisely. You are not using a hammer as your only tool sometimes you need a spanner.

Agility – Its lighter and possibly more agile as your development can be adapted to suit your goals as they change and develop.

Speed and efficiency – you can spend a lot of time bending an existing package to your will. To the point at which you almost break it.

Faster – often your not carrying a pile of *features* which you never wanted but got as a side effect of the package you adapted.

Easily adapted – more easily adapted with the kind of features that come up client demands, specific legislation or business focus changes.

Intuitive – usage can be designed to flow in a way that fits with existing systems of thinking and terminology so it becomes a lot more intuitive.

Unique to you – hopefully your organisation does something unique, your competitors can also use off the shelf software so are you removing a possible advantage over them by using the same.

Directed – Start with what you want to do and go from there no compromises.

Licensing – you own it a bespoke development is generally going to avoid on going licensing costs that may be associated with existing packages.

So for your untypical problems and challenges perhaps you need a untypical and bespoke solution.

Can I eliminate all errors?

Trying to move towards a zero tolerance for errors.

  • Even minor errors can obscure more major ones
  • Any error could be cancelling out another error – a kind of error ridden hell to avoid at all costs
  • Errors that seem insignificant now may be come more major issues in the future (when deprecated becomes removed for example)
  • Broken windows – lead to more broken windows.
  • Errors annoy me
  • Its hard to take any pride/pleasure in code that has even minor errors.
  • Every error is an opportunity to improve code
  • The cost of dealing with errors later is likely to be more
  • Errors can be an opportunity for a hacker to gain information about your system
  • Every error is an opportunity to learn – if you had understood it completely before that error would never have happened.
  • Errors cascade and create other errors

Its better to preempt problems before they occur in the wild. Users shouldn’t have to deal with errors or experience their side effects. Its better for the clients site and whats good for them is good for me (eventually). Its probably impossible to eliminate all but you can squash all the ones you know about. At least with every site reporting all errors however minor to a central source, any that occur over night can be sorted out by breakfast.

Having said that going through 404 logs does make you paranoid. There are bots out their trying to get you :). No really there are.

When its all gone wrong

Sometimes your relationship with your developers can go wrong. What happens if they won’t do anything or won’t enable you to update your site?
Here are some terms its helpful to understand when sorting out the mess. Then some pointers on moving your email and moving your website.

DNS

DNS is the system that tells the internet where to look when someone types in your domain name.
Say your companies website is at www.example.com. When you type www.example.com into your web browser, your computer does a search and finds the IP address for that domain name. Its like sign posting system. This makes it quite crucial, if you can get control of your DNS you can point visitors to your site and emails to your anything@example.com to a new provider.

DNS records are a complicated subject but there are two main types

  • MX records – where your email is
  • A name records – where your website is

IP Address

The address of your server as seen by your computer so like 192.0.43.10 or 2001:500:88:200::10: pointed to by your DNS.

CMS

Content management system – the system that allows you to edit your site examples include WordPress, Joomla, Drupal, DotNetNuke, Umbraco, Django, Mura, etc, etc.

Moving your email

The things you will need to know to move your email to new provider are really just the list of email addresses. If you have the existing details (password, username, incoming server) that is ideal as you can use them to pull over any email from the old server.

Moving your website

This is a bit more variable, if you have a static site. It is just a bunch of files all of which are publicly available. So you can grab them or your friendly developer or hosts can.
If you have a dynamic site that has a database and some server side languages it is more difficult. For these the source code isn’t the same as what you can see so you have more difficulties. These sort of sites usually have pages that end with extensions other than .html or .htm for example (.php,.aspx,.cfm) or sometimes with no extension at all like example.com/my-monkey-submarine.

You can make a static copy of a dynamic site as you would for a static site but you do lose any CMS and the information that was in your database that wasn’t publicly viewable.

The vast majority of the time its not like this your existing developer will give you the raw files and database (that is the ideal case). Its your information, you paid for it and you should own it.

Your super idea

Everyone has brilliant ideas in the bath sometimes they even look good in the daylight. (see Archimedes)
You may be a bit hesitant to tell anyone about your idea. Here are some simple things you can do on your own to check out its viability.

  • Do some Google searches around your idea, is anyone else doing something similar?
  • Not to worry you don’t have to reinvent the wheel just have a different spin on it most businesses aren’t revolutionary more evolutionary.

  • Try it out on a few trusted friends, your spouse, partners
  • Have a look at domain names are they available for your idea, if people are using a similar name to what your thinking are they doing a similar thing to your idea.
  • Find someone who could be a potential client/customer ask them what they think.

Telling other people about your idea is a quick way of trying stuff out. You evolve your idea just by repeating it to people and gauging their reaction. Are they getting it or are they struggling to understand what your going on about.

You probably think everyone is going to run off with your precious new idea, to be honest they are more likely to just forget it. Also any enterprise is a lot more than just the idea, the execution is the majority of it. Did Microsoft become dominant because the were the best at original ideas or because they were good at execution?

Then you could try prototyping your idea, build a little prototype website only a cheap thing just a day or so in cost perhaps. Hold a test event. See if anybody is interested, really.

Cyberdojo

Went to a cyber dojo last night at SyncNorwich run by Jon Jagger who had come all the way from Somerset. Its quite an interesting idea you pair up on tasks and just practise. You can give it a go on Jons site here. We paired up on java exercise which was a laugh as Chris and I had no experience in java at all, so we spent quite a lot of the time working out the syntax.

Jon was more focused on the how and why of the process, not writing more code than you need, working together. He summed up and you could see in the review at the end different peoples approaches and stuff like the number of tests each pair had written. Their patterns of test fail/passes and each pairs process step by step.

Jon was saying that some companies had used as part of an interview process pairing there regular developers with candidates and rotating partners gave the candidates a good idea of the company and the people. On the other side the company gets a good idea of how the candidates think and how they are to work with.

Jon started off by asking who used to code when they were kids was it fun – lots of hands. Of those of you who code for money now when was the last time you wrote coded for fun? not many hands. Bit sad really, hopefully cyber-dojo could bring back peoples Joy in Code.

Fuzzy testing

Been trying to work out what browser based testing is good for and not good for. Went to a talk to the other night at syncnorwich about testing at google scale. Seems they do use browser based testing but more as a top level of icing above a lot more tests of smaller units of functionality at code level.

It seems to make sense to test things as they are seen by the client, so I’ve been poking at Watir which allows for remote control of web browsers and therefore integration into automated tests. Apparently its jut one way of going about doing this and its a thin wrapper around selenium and web driver. I’m currently also interested in Ruby though so I went with watir (pronounced water apparently).

it seems browser based testing is good for a few use cases

  • Different browser
  • End to end tests
  • Random inputs
  • Pounding on a url with random urls

It seems like probably browser based testing is good for end to end overview testing. Making sure that even though all your unit tests work the web application works fine when all put together.

Daniel Wagner-Hall mentioned fuzzy testing to me which seems an interesting idea and something easy to automate but its not something I’d heard of before.
So I started on wikipedia

“Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.”

This seems like something that watir would be good for pounding on a page/site with random form inputs testing out back and forward browser actions, clicking random links etc. Cat walks across the keyboard style. The kind of thing a monkey would be good at one of the monkeys that aren’t busy typing the complete work of shakespeare. Never really understood why would a monkey would want to type shakespeare surely monkey fiction would be quite interesting?.

So anyway I’ve given my monkey something more interesting to do. You will need to have Watir installed/setup and I’m using Firefox but you could swap that out for your browser of choice.

web monkey script

5 Why’s

Or why acting like your 5 might be a good thing

  • Child: Why is the sky blue?
    Parent: I don’t know
  • Child: Why don’t you know
    Parent: because I didn’t get taught that at school?
  • Child: Why didn’t you get taught that at school?
    Parent: because I didn’t pay attention
  • Child: Why didn’t you pay attention?
    Parent: because I was naughty
  • Child: Why were you naughty?
    Parent: because I didn’t eat my crusts

If you ask enough questions you get to the root cause and may end up solving the root cause of the issue rather than covering over the cracks or resolving symptoms of a greater problem. Also a greater understanding of the issues around the problem can help next time you attempt something similar or are planning for the future.

Issues

I think its ok to ask that many questions you get loads of input/info which is always a good thing. You have to be a bit careful how you phrase it perhaps not quite so bluntly is a good plan. The person your asking can take it the wrong way ‘you think they are wrong’ rather than you want to ‘find out and understand as much as possible’.

On the downside you do need to have some knowledge around the subject or you struggle to ask relevant questions. Also you need to know enough to know if you being fobbed off with a ‘eat your crusts’ style answer. Then you can ask the last question

‘Why does not eating crusts make you naughty?’

Sacred cows

You should also question the sacred cows, sometimes they just hang around because nobody questioning them. If they are legitimate they will only be made stronger by a bit of polite whying. There isn’t anything sacred about the number five its more a rule of thumb, you often only need one or two questions its the getting to the root cause that is the important thing.

Sometimes the root cause isn’t solvable and you just have to work back to the things you can solve. In The Lean Six Sigma Pocket Toolbook the authors give an example of the photocopier jamming.

  • the copier jams because the room has high humidity
  • why does high humidity make the paper jam?
  • because the paper absorbs the moisture.

Then you can go back to the problem you can solve – reducing the humidity in the room.

Digging for Requirements

Its also a useful idea when planning a project. In the Pragmatic Programmer they have a section you shouldn’t gather requirements you should dig for them.

‘Requirements rarely lie on the surface. Normally, they’re buried deep beneath layers of assumptions, misconceptions, and politics.’

Its much better than starting off building something on a false assumption.

As my friend Tim says assume makes an Ass out of U and Me. Curiousity about lots of things makes the world more interesting.