Tag Archives: testing

Cyberdojo

Went to a cyber dojo last night at SyncNorwich run by Jon Jagger who had come all the way from Somerset. Its quite an interesting idea you pair up on tasks and just practise. You can give it a go on Jons site here. We paired up on java exercise which was a laugh as Chris and I had no experience in java at all, so we spent quite a lot of the time working out the syntax.

Jon was more focused on the how and why of the process, not writing more code than you need, working together. He summed up and you could see in the review at the end different peoples approaches and stuff like the number of tests each pair had written. Their patterns of test fail/passes and each pairs process step by step.

Jon was saying that some companies had used as part of an interview process pairing there regular developers with candidates and rotating partners gave the candidates a good idea of the company and the people. On the other side the company gets a good idea of how the candidates think and how they are to work with.

Jon started off by asking who used to code when they were kids was it fun – lots of hands. Of those of you who code for money now when was the last time you wrote coded for fun? not many hands. Bit sad really, hopefully cyber-dojo could bring back peoples Joy in Code.

Fuzzy testing

Been trying to work out what browser based testing is good for and not good for. Went to a talk to the other night at syncnorwich about testing at google scale. Seems they do use browser based testing but more as a top level of icing above a lot more tests of smaller units of functionality at code level.

It seems to make sense to test things as they are seen by the client, so I’ve been poking at Watir which allows for remote control of web browsers and therefore integration into automated tests. Apparently its jut one way of going about doing this and its a thin wrapper around selenium and web driver. I’m currently also interested in Ruby though so I went with watir (pronounced water apparently).

it seems browser based testing is good for a few use cases

  • Different browser
  • End to end tests
  • Random inputs

    • Pounding on a url with random urls

It seems like probably browser based testing is good for end to end overview testing. Making sure that even though all your unit tests work the web application works fine when all put together.

Daniel Wagner-Hall mentioned fuzzy testing to me which seems an interesting idea and something easy to automate but its not something I’d heard of before.
So I started on wikipedia

“Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.”

This seems like something that watir would be good for pounding on a page/site with random form inputs testing out back and forward browser actions, clicking random links etc. Cat walks across the keyboard style. The kind of thing a monkey would be good at one of the monkeys that aren’t busy typing the complete work of shakespeare. Never really understood why would a monkey would want to type shakespeare surely monkey fiction would be quite interesting?.

So anyway I’ve given my monkey something more interesting to do. You will need to have Watir installed/setup and I’m using Firefox but you could swap that out for your browser of choice.

web monkey script

Mum Testing

Just been trying out a new site on my Mum’s friend.

Doesn’t need to be your Mum any other user is fine as long as they match the profile of the users you expect to see. I’m trying the site out on my mum and my mums friends because that is the age group we expect the site to be most used by. I’ve built sites before for school age kids and we tried it out on them and their teachers.

Part of it is just to get a new view, often you get so used to how something works ‘because you built it’ that you forget how other people use the interweb. Your friends often have similar levels of familiarity with the technology as you so use things in the way you would expect. Mums are quite good because they don’t use things how you do, just sitting behind someone using your site/application can teach you a lot. The technology they use may be different as well you might use a nearly new massively monitored Mac and they are using a venerable small screened PC. Or perhaps they are part of the majority of the population who use something other than an iphone. That can be quite revealing as well. Much of the benefit though is just the first view effect, they like perhaps the majority of the visitors to your site are seeing all this for the first time. If your tester can’t figure something out its probable a large group of your potential customers won’t be able to either.

Example

For example I watched someone fill in a form and miss out the email field, I thought I’d made it really clear the field missed changed colour to make it stand out as it did its label and its border. It even added a block of text at the top of the form explaining that the field had been missed. The tester didn’t notice any of that so I added in a movement when you clicked submit the page scrolled up to the missing field to draw you attention to it.

It also brought up another issue for me though as I found myself saying “don’t worry you won’t get any junk emails”. That led to adding a little note at the top of the page to clarify how the details you enter will be used.

Different folks

Different groups of users may have different levels of skills and familiarity with the subject. Something that may make a lot of sense for school teachers won’t may not make sense for others, but if your customers are school teachers then that is fine.

You need a few people though just to get a range even if the first couple of people show up the vast majority of the issues. Users do the craziest things which is explained quite well by Steve Krug in “Don’t Make Me Think!” its not consider page in depth its more a smash grab, super market sweep style. They almost certainly won’t use your site in the way you imagined.

A book that got mentioned by Tarek at my local Ruby user group when we were talking about human machine interaction and the assumptions that users bring is ‘The Design of Everyday Things’ by Donald Norman. I have not read it yet but it sounded like it could be useful.