I have seen a couple of authors "assume" information not explicitly written, and another tester could not duplicate the results. Assess the probability of this type of incident occuring before deciding who tests.

Authors know they can easily overlook unclear instructions or a missing detail, so they seek a good editor on the project team. Most reviewers/approvers want only to review content and not edit or help authors clarify. (I think authors should revise sentences that evoke reviewers' questions.) To me, the best editors are intended users (trained on the system but have little experience using it). They do not mentally insert steps or details that are not written. If a user is not available, and another team member can't or won't edit, then I feel it important to have another person (not the author) execute the OQ scripts. Minor edits required during execution are not protocol deviations (e.g. adding a word, phrase, or short sentence).

Connie, thank you very much for your feedback. So you recommend peers review prior to test script approval to overcome the changes you described.

Yes. It is a collaboration among peers. Any peer can make comments, corrections, additions, etc. We "always" use the Tracking feature in MS Word so people may accept or reject the changes.

Test script shall be prepared in a way that anyone other than author can execute by going thru the test cases.

If team is small, we will be getting to this situation that author executing his/her test cases.This may lead to conflict when he/she involved in the development of the work being tested which wouldn't be the case when team have dedicated testing team.

While incase of unit testing , normally developer only will test his/her part while peer review will be followed to assure the quality.

As commented by Connie, build a process of peer review of test cases(Tester, Developer , BA and Lead) to ensure it's meeting the standard.

As long as there is a strict review by lead or another team member, the author can perform the testing .

. . . what Connie said . . .

Agreed with the comments above. I'd also look to see how your company handles laboratory method validation processes to see how they operate (and to be consistent since the concept is the same). I'd bet the person who wrote up a method validation protocol is also executing against it after the appropriate peer reviews.

You could have an observer that would ensure the author follows the script as written.

I believe that if the author is separate from the development team then it meets the spirit of testing. Another person with knowlegeable experience should be able to reexecute. I do also think there needs to be an independant review/approval of the final execution results.

Conflict of interest. Your QMS will not survive a QSIT inspection.

Maurice Levie, would you care to elaborate on your statement?

A QMS that lacks rigor around the right side of the "V" isn't worth the paper it is written on. Maintaining the integrity of the design validation through rigorous testing is a must. There's a $100 book on the do-nots of validation, and the majority of the cases deal with testing. Your PQ testing means little if the OQ and IQ testing phases are compromised.
Feel free to reach out to me on the topic of quality center/ALM as a solution to paper/non-automated testing

If you are FDA-regulated, then having an author execute their own scripts is, indeed, against best practice and also the Guidance for Software Validation. If you are not FDA-restricted, then there are more lenient practices, like this one, that are accepted. However, in my experience of over 20 years, if resourcing allows, I will not allow the author to execute. I have a strong belief that the author gets too focused and cannot see the ancillary testing that may be needed. Being reviewed by a peer and QA does not satisfy the detailed review needs, as these two parties are often not involved in the specific details of the product until the review process, leaving them at a disadvantage due to lack of knowledge of the intricacies of the product behavior and also how the users actually use the product. I do require a dry run of all scripts prior to the official testing, where the author or peer actually perform the informal testing by the script without documenting results. This dry run is done in order to catch anything that needs updated on the script itself. Otherwise, there will be defects for the script errors.

Author executing own test scripts is a no-go. Author may be biased.
If the author would also be the executor, then a deviation report should be written.

OQ test script author shall know the business processes and the Requirement Specification/Functional Specification and write the OQ test script based on this.
Executor shall have no interest in passing the test with neither system errors nor script errors. If the system did not meet the requirements of the RS, then this shall be evident after OQ test script execution. Therefore executor can neither be the developer nor the author.

It is important to stay RS compliant through all parts of the V-model.

The FDA General Principles of Software Validation leverages concepts of 'independence of review (i.e. author can not approve their own work).' However, I haven't seen any regulation, or guidance (GAMP testing guide accurately states authors shouldn't review their tests prior to execution, and testers not approve their own work) explicitly prohibiting an author to execute their own protocol or stating this is a conflict of interest, but I may be incorrect. Internal procedures may dictate otherwise. Again, I would look to see how your org handles this concept outside of the world of csv (i.e. method validation).

I have practiced the principle of Test authors should not execute their own test scripts based on "Independence of Review" principle and some organisations defining this in their procedures. However I know few projects we have only single member in Testing team who authors all test cases and execute all of them. but as a best practice if more than one tester in project should ensure tester and test author is different for a given test case - this is only a best practice though not a regulatory requirement as long as you define procedure accordingly.

However regulation does not dictate it explicit. Hence from regulatory perspective organisation has freedom to define and be compliant. One need to author and other need to review and another approve test scripts before execution. Now the tester will execute after the test scripts are pre-approved.

However, if project proves defects are passing to next stage because of test author and tester is same, update procedure as CAPA

1. As Akash Arya says, I have not seen any regulation prohibiting the author from testing.

The regulations clearly state the desired results (e.g. data integrity, security, trained persons) and not "how" to obtain those results. Guidances stated that regulators do not want to restrict how to use new technologies that did not exist when the regulations were written.

2. As Crystal Risotti says, the availability of resources is a key factors. If resources allow, then have a second person test and don't let the author test.

In small laboratories (e.g. 5 employees), there may not be enough people to have a separate person test.

3. As Cindy Shank says, the author can execute if you have an observer 'ensure the author follows the script as written." The observer must watch keystrokes, watch mouse/cursor movements, and sign the executed test script along with the tester.

Another alternative: outsource writing the document, then have an employee test.

We should not start to over interpret regulatory requirements.
> Test scripts must be peer reviewed and approved prior to test execution.
> After the test execution, the test results must be reviewed and approved.

As long as reviewer and approver are not the same persons as author and tester, I cannot see any formal trouble in terms of segregation of duties.
If the people are adequately trained and honest, even a developer can write a test script and execute it.
However design review is a team work and design failures could be identified by the team (SME, QA, etc.).
Based on my experience, I do not rely on and believe of test witness. In many cases, the observer is only an alibi but it does not really enable a quality improvement.

Finally, everything depends on the quality ethic of the organisation not on the number of involved people.

Yves, thank you very much for your comments, I agree with you, we must think out of box (past best practices) and look for practical solution. The bottom line is we should evaluate the risk associated with the decisions like this.

Thanks you every one for your comments. It helped me a great deal. I will soon share what we plan to implement.

I DO allow testers to execute their own scripts, BUT only on the basis that (a) their test case is independently reviewed (peer reviewed as a minimum) prior to execution, to ensure that it is an appropriate test to challenge/confirm the requirements and (b) that their is independent review post execution, to verify that the evidence demonstrates that the test objective was met. This meets the regulatory expectation for independent review, while also allowing someone who knows what they're doing to test the software (but I also encourage unstructured testing, which finds as many issues as structured testing). What isn't acceptable is a tester testing their own software with no effective review.

I would like to ask: what if you just have a short time project with a one-person team, which I have seen, being the one person. Yes, reviews and approvals were required by other roles, but we called that out in our VP and summary reports and there were no regulatory repercussions. its the old saw: good, fast, cheap. now pick two of the three.

There should be segregation of duties established to control testing. However it depends if it is regulated or non-regulated system and risk assessment of the requirements.

We agreed to insert the statement in our V&V plan. Author of the test case can execute their own test case. Peer review of each test case will be performed by a team member other then the author of the test case. Peer review may include dry run of test case. Each test case shall be approved by functonal and QA representative as required by V&V Sop referenced in this document.

Thank you all for your comments.

I don't like the authors to execute their own test. Very often they played around with the system before and now reexecute, what they have already seen before, what hardly adds any value. They just run the system how they expect it should work, whereas another tester may run the system in a different way ( of course depending on the detail level the test is described), what significantly increasrs the "succes" of error detection.

I have seen this done before but like Thomas said the script writer will know how to interpret the protocols more than a different person. Some might consider it a conflict of interest but it you are following your procedures and have QA's blessing then you are OK.

Honesty is one of the major properties!
I would train the testers (whoever the testers are beside: developer, user, script writer, etc.).
In such training, I would emphasize the importance of honesty. We should not blame the testers if they find failures.
Relying on the various presentations this week in Copenhagen, in particular such provided by David Churchward as well as Nuala Calnan, we have to care of the fraud triangle: opportunity, rationalization, and pressure.
We need to educate our testers for ethic since the final purpose of testing is to demonstrate the fitness for purpose of the tested equipment/application/... including - risk-based - its reliability and capability.
Failure discovery and reporting are part of the testing business.

Dear Yves - thank you for your very rational response to a good question (one I have reflected on myself on several occasions). This will be very helpful to me.

The adoption of theories (in this case financial auditing) from one discipline and ready acceptance by another (DI - the subject lectures) is one way that disciplines grow their knowledge and is readily accepted with study. The Fraud Triangle (FT) theory first proposed in the 1950s may apply to DI when and if those studies prove successful. However, the application FT, in this case, to a tester procedural matter, or even a regulatory matter, is an invalid application of the original work. Even if one tries to extend the FT to include the role of the test developer in potential fraud the conclusion can be made from the discussions above that this is a work force shortage issue not an attempt at monetary gain.

@G.M. Butcher:
In the particular case, I am not considering the FT in terms of avoiding financial fraud, but I think that the missing tester accuracy and honesty could derived from the areas of concerns mentioned by the FT.
Business pressure, missing collaborative management, inappropriate KPI could cause tester's misbehavior.

We do allow the authors to execute the script, however it is peer reviewed and critical data entries are also verified by an independent person. Quality then post approves the scripts. Remember, if you execute, you cannot review or approve, if you review you cannot approve and if you approve, you cannot execute or review.

@ Yves Samson - Note this side discussion is off topic and I do not like to promote that, so brief comments. 1. My original point was; one cannot adopt a theory without studies that show it is applicable in a different field. Where are your data? 2. Original use of FT was based on monetary gain, so until there is data showing other reasons to apply the FT theory, work force shortage was/is the issue whether viewed as a procedural or regulatory matter. Diversion ended.