Software Testing

[domcobb180]

How does Contract Software Testing work : as a perminant employee I can test software and give recommendations but it's an art not a science. Full testing is impossible unless you have a ridiculously simple system so there are always issues missed by testing/QA and potentially live issues as a result. (Two testers could test the same system and come up with two sets of bugs/issues...) - How do seasoned contract testers feel about this and how they find their contracts/work out in practise? - In many organisations, testers are come under fire from project teams and are sometimes held up as scapegoats for project issues (last minute changes rushed through with no time for reasoned regression testing etc).

Is contract system testing any worse than perminant and are there any particular issues with contract testing that perhaps don't apply to other roles? I could envisage that testing is perhaps more risky than say programming from a blame/liability perspective?

Is it just a case of phrasing the contract to reflect that services are offered on a best endeavor basis and specifically exclude any liability for direct or indirect losses/consequential losses arising from the services provided in addition to taking out the necessary PI and limited company setup?

[stek]

Places where I have contracted generally have several test environments of increasing complexity to do full end to end testing, functional and performance and volume. Is this the scenario you work in?

[doodab]

The secret is to present a truly comprehensive plan. Then, when it's vetoed and corners are cut on grounds of time and cost you have your excuses for anything that gets missed.

If the plan isn't vetoed then there is a good chance the system will be replaced before it ever gets switched on.

[domcobb180]

yep current place is pretty structured in terms of test phases - seperate functional system testing, non-functional and UAT done jointly with the business users. But still issues get through to live every so often.

[Freamon]

yep current place is pretty structured in terms of test phases - seperate functional system testing, non-functional and UAT done jointly with the business users. But still issues get through to live every so often.

Are these issues cases where the product does not meet the business requirements (as documented)?

Or where the product does not exhibit a certain (previously documented) behaviour?

You can only test against the requirements/behaviours that are specified.

If the issue is some unexpected behaviour that the customer notices in live, but does not contradict any of the previously documented requirements/behaviours, then testing isn't the problem.

(A good way to spot this kind of problem early on is to review the specified requirements/behaviours to ensure they are written in such a way that it is very obvious exactly how you would test the product against them).

[AtW]

I enjoy failing developers when testing their tulipy software.

Also I love slurping coffee.

Mich the Bastard.

[domcobb180]

Are these issues cases where the product does not meet the business requirements (as documented)?

Or where the product does not exhibit a certain (previously documented) behaviour?

You can only test against the requirements/behaviours that are specified.

If the issue is some unexpected behaviour that the customer notices in live, but does not contradict any of the previously documented requirements/behaviours, then testing isn't the problem.

(A good way to spot this kind of problem early on is to review the specified requirements/behaviours to ensure they are written in such a way that it is very obvious exactly how you would test the product against them).

Agree in theory but in practise you find on a lot of projects this isn't always the case and you have to roll with a sub optimal documentation set or a set that's fairly high level with a lot of ambiguity - is it practical to refuse to work on test cases until a set of perfect or near perfect documentation exists.... As a perm you can negotiate on this but mostly there's a compromise. How do contract testers find this goes down - is it a case of downing tools until a perfect doc set is produced? In addition, you can test something and it works with one dataset or at one time slot (or in conjunction with other processes) but won't work with another but to test all possible data instances/process combinations would be impossible. Then you'd end up with a requirement failing under a certain context but probably out of 100 testers, none would find the issue or think to specify anything other than a fairly generic test case.

[eek]

Agree in theory but in practise you find on a lot of projects this isn't always the case and you have to roll with a sub optimal documentation set or a set that's fairly high level with a lot of ambiguity - is it practical to refuse to work on test cases until a set of perfect or near perfect documentation exists.... As a perm you can negotiate on this but mostly there's a compromise. How do contract testers find this goes down - is it a case of downing tools until a perfect doc set is produced? In addition, you can test something and it works with one dataset or at one time slot (or in conjunction with other processes) but won't work with another but to test all possible data instances/process combinations would be impossible. Then you'd end up with a requirement failing under a certain context but probably out of 100 testers, none would find the issue or think to specify anything other than a fairly generic test case.

So you write you test specification based on the incomplete specification and record a brief set of areas which cannot be tested due to ambiguities within the specifications.

As for data sets and process combinations that is what automated testing is for. schedule to work through all the variations you can think of and ensure they work.

[EternalOptimist]

The infratructure dudes here finally set up a test area at ClientCos client, yippee, we now have a test system. three months after the system went live with bugger all UAT

[eek]

The infratructure dudes here finally set up a test area at ClientCos client, yippee, we now have a test system. three months after the system went live with bugger all UAT

It went live so wearing my consultant hat that's sign off. Everything is now a chargeable change request (even if it is a bug).