Mitch wouldn't?

[vetran]

Crowdsourced flaw-finding cheaper than in-house bug hunters ? The Register

A study into the once-controversial practice of vulnerability rewards programs (VPRs) – paying researchers bug bounties for reporting security flaws – has found that for browser builders, the practice is not only more effective at spotting problems that hiring code-checkers, it's also much better value for the money.

---------------
So Mitch have you seen this done(close up)? What are the issues?

[Mich the Tester]

Crowdsourced flaw-finding cheaper than in-house bug hunters ? The Register

A study into the once-controversial practice of vulnerability rewards programs (VPRs) – paying researchers bug bounties for reporting security flaws – has found that for browser builders, the practice is not only more effective at spotting problems that hiring code-checkers, it's also much better value for the money.

---------------
So Mitch have you seen this done(close up)? What are the issues?

Yep, I've seen a lot of crowd testing. It has advantages in being cheap and quick and in gathering information about the real use of an application. However, it also exposes your bugs to the world for the competition or for dishonest people to know about, and the coverage achieved by crowd testers can't be meaured. That's OK if it's a simple app that isn't critical and with continuous delivery you fix the bugs quickly anyway. But once the application is more critical and subject to legal or hard security requirements it's a different matter. Would you want security critical stuff like airport baggage scanners or emergency services systems (the kind of stuff I work on) to be 'crowd tested'? Do you trust the crowd to report the bugs they find instead of abusing the knowledge they gain? Can the crowd also help repair the bugs or even prevent many of them being created?

Crowd testing is a good thing where it's sensible to apply it and it has a big future, but unfortunately some managers seem to think it's a silver bullet. Good for quick delivery of simple apps that can make money but are not critical, but not a good idea for stuff that people really have to rely on for their lives.