How do you conduct performance qualification testing and review for a cloud-based computer system?

PQ assures that the system meets the indented use i.e. requirements defined by the users of the system. PQ test plan shall be written to define the strategy of testing, test environment, acceptance criteria, roles & responsibilities etc. Test Plan shall also define the test data. Test data shall be representative of the actual data that the cloud-based system will process. Testing strategy shall include scenarios that simulate the real-world use of the system. Thes scenarios shall include wide range of use cases including stress testing. Test plan shall include any pre-requisite that needs to be fulfilled before doing the PQ testing including trainings to be done by the testers.

Performance qualification testing for a cloud system involves checking how fast and well it works under different conditions. This means simulating various scenarios to ensure it meets your needs. Periodic reviews mean regularly checking if the system still works properly, is secure, and stays up-to-date with the latest standards.

In my experience, we always strive to involve the WHO of PQ. In most cases, it is the actual business users - to getting them engaged early is the key. Think of Process exercises, demos etc to get them familiar so that when the time comes to execute PQ/UAT, they understand and can complete the test efficiently. This also will allow to think future cases and will encourage users to include such test scenarios into the PQ plan and execution.

Strategy for PQ should be defined in the Validation plan. Test cases should have pre-defined acceptance criteria. Test cases from OQ should not be repeated. PQ should only verify the performance of the system and not functionality tests (functionality tests should be covered under OQ).

In my experience, a common challenge in designing PQ test cases for cloud-based systems is ensuring comprehensive coverage of potential risks, particularly those related to latency issues and non-federated services. To overcome this, we incorporated these specific scenarios into our risk assessment and designed our test cases accordingly. This approach, grounded in real-world user experiences rather than a simple repetition of OQ tests, significantly improved the relevancy and effectiveness of our performance qualification testing.

✅ Start with an approved Test Plan and Test Cases, meticulously covering all URS requirements. Tailor your extent and effort according to identified risks. 🎓 Equip your testers with thorough training on the Test Plan and Test Cases. Ensure execution adheres to Good Documentation and Data Integrity protocols. 🛑 Encounter a deviation? Raise a validation deviation/defect promptly. Investigate root causes and conduct retesting as necessary. 🔍 Review executed test cases meticulously for accuracy, completeness, and consistency, ensuring reliable test results. 📄 Keep all documents under version control, facilitating seamless future reference and traceability.

PQ test case results should have evidence attached. Evidence should follow ALCOA principle. Any deviation or out of specification result should follow validation incident reporting as per company procedure.

One thing that I've found helpful and require from my team is separation of duties. After you PQ test execution is complete, have a quality team member review it for accuracy, completeness and to ensure that it follows your company policies, procedures and guidelines.

Planning around designing PQ test cases in my opinion and experience is extremely crucial for ensuring quality of an IT system. Taking into consideration every possible positive and negative scenario for testing ensures that there is no room for uncertainty or unknown outcomes when the system enters production. Documentation with versioning helps keep things organised. TIMS and Azure DevOps are great examples of maintaining testing documentation.

🔍 Once the PQ test cases are executed, reviewed, and approved: 📝 A PQ test report is prepared. 📊 The report summarizes test results and identifies defects, if any. 🕵️♂️ It undergoes review by relevant stakeholders, including the Quality team. 🎯 The PQ report concludes with a decision regarding the system's release for usage. #ProductQuality #Testing #StakeholderReview

Executed PQ or PQ report should be reviewed by testers and QA. Report should be cross-checked with against URS and functional specifications to ensure all the requirements are met.

Systems / Software should undergo periodic reviews. Frequency of periodic reviews should be defined in the procedures. Frequency should be based on the GAMP category of the system, and impact of the system on business.

In my experience, maintaining cloud-based system performance is a continuous process that requires constant vigilance. Utilizing native services provided by the cloud provider, such as AWS CloudWatch or Azure Monitor, can significantly aid in monitoring system performance, usage, and costs. Additionally, these tools can help identify unauthorized access attempts, including those from regulatory inspectors. Regular audits and reviews are crucial to ensure ongoing compliance and system effectiveness. Any system changes should be thoroughly documented and their impact on system performance and validation status assessed. AWS, Azure and GCP have many built-in services to do performance monitoring and parts of periodic review automatically.

One of the best ways to understand Cloud-based CSV is to understand where the risks are in the Cloud. Information security is one of the major risks because the cloud datacenters are completely managed by the vendor. What about disaster recovery and routine backup and restore testing of the system? With so much risk to the system shifting to the cloud provider, have you performed a thorough Vendor Audit of these high-risk areas to understand what you need to do as a company to be compliant with regulatory guidelines? Did you know that Azure, AWS and GCP all have compliance documents available at their cloud web sites?