DEVOPS, SELENIUM, AUTOMATION, TESTING

Automated User Acceptance Testing (UAT) using Selenium IDE within Clinical Trials

DevOps Automated UAT

User Acceptance Testing (UAT) is a critical component in the clinical trial process. In the article I have provided an introduction to using the Selenium IDE application as a method of automate user acceptance testing.

If you want to skip to the set up and testing using selenium go here.

Clinical Trial Data Management (CTDM) is a critical phase in clinical research, which tried to ensure the generation of high-quality, reliable, and statistically sound data from clinical trials. CTDM also ensures collection, integration, and availability of data at appropriate quality and cost.

Clinical trial software testing main purpose is for regulatory compliance. There are many testing methods which can help to ensure and reassure compliance of software used in clinical trials, this includes:

  • Unit testing
  • Functional testing
  • Usability testing
  • Performance testing
  • Load and stress testing
  • Data integrity testing
  • Security testing
  • Failure and recovery testing
  • Configuration testing
  • Regulatory requirements testing
  • Regression testing
  • Integration testing
  • User Acceptance Testing

The testing that will be focused on for the remainder of this article will be User Acceptance Testing.

Can user acceptance testing be automated?

User Acceptance Tests (UAT)

User Acceptance Tests (UAT) is the typically one the last steps when developing a particular feature. The application is used by people from the intended target audience in scenarios which will be typical to the real world usage (or as close as possible). The users will record and note any defects which are discovered. It gives end users the chance to interact with the application and find out if everything works as was intended. This process should help discover aspects of features which may have been overlooked, miscommunicated, or are defective etc.

Who is responsible for UAT testing?

In my current organisation, the responsibility of User Acceptance Testing (UAT) is the responsibility of the end users (along with guidance from development team).

Can user acceptance testing be automated?

Automated User Acceptance Tests (UAT) are something that very few end users and quality assurance (QA) teams actually do.

Why don’t more teams automate acceptance tests?

One main reason I have experienced that more teams don’t automate acceptance tests is because of:

  • Lack of knowledge or skills to implement
  • Not allocating enough resources

Why should we try to automated our User Acceptance Tests:

  • Reproducible - Repeat what was done if audited years into future
  • Traceable - Map tests to requirements
  • Evidenced - Developer and user testing

MHRA / GCP

Medicines and Healthcare products Regulatory Agency (MHRA)

Good Clinical Practice (GCP)

  • Appropriate controls of the system are in place throughout the systems lifetime
  • The system is fit for purpose and performs reliably and consistently as intended
  • Computer Systems should be periodically evaluations to confirm that they remain in a valid state and are compliant. This should include:
  • Current range of functionality
  • Deviations records
  • Incidents
  • Problems
  • Upgrade history
  • Performance
  • Reliability
  • Security
  • Validation

References [14.5.1 Validation Principles. MHRA Grey Guide]

[EudraLex The Rules Governing Medicinal Products in the European Union Volume 4 Good Manufacturing Practice Medicinal Products for Human and Veterinary Use Annex 11: Computerised Systems – Equivalent to FDA 21CFR11]

3 Step Process

How should test be conducted:

  • Arrange (Setup)
  • Act (Perform Actions)
  • Assert (Check Expected Result)

Testing Work flow

  • User Requirements Specification
  • Functional Spec (Implementation + Risk)
  • Test Scripts based on risk (Collaborative)

Development Work flow

Development Setup

  • Dev - local (laptops/work stations)
  • Test - unstable code, seeding data
  • Stage - pre-production
  • Production-live
  • Production-test

Selenium IDE

This section comprises from my notes into looking into the selenium IDE plugin for google chrome/firefox for the purpose of automating our user acceptance tests.

Selenium Installation

The application we are going to use in order to help us automate our user acceptance testing (UAT) is selenium.

https://www.selenium.dev/selenium-ide/

In particular, we are going to be using the selenium ide which is available as a google chrome and firefox extension.

Download and install selenium browser plugin for your browser of choice from this website.

selenium website download

https://www.selenium.dev/selenium-ide/

Google Chrome

https://chrome.google.com/webstore/detail/selenium-ide/mooikfkahbdckldjjndioackbalphokd

browser chrome

Firefox

https://addons.mozilla.org/en-GB/firefox/addon/selenium-ide/

browser firefox

Once you have installed the web browser extension, you can proceed to open the extension.

selenium chrome bar

You will be greeted with the following menu. The options are to:

  • Record a new test in a new project
  • Open existing project
  • Create a new project
selenium menu

We are going to setup our first test to get used to setting up projects and tests using the web recording feature of the selenium plugin.

We open the selenium IDE, and select Record a new test in a new project Given the project a name, I use TestProject

You then need to specify your base url. This is essentially the starting point we want our test to start from. I use http://www.google.co.uk. Note, you need to specific the http:// part of the url, otherwise selenium will not let you click the Start Recording button.

We then click start recording. This will open the web page we previously specified, and start the recording. Once the recording has been started, all actions we perform in the web browser will be recorded. An active list can be seen in the main selenium window as you perform each action.

selenium window action

You can also note from your web browser window that selenium is recording your actions

selenium record notification

Once your test is completed, then you can save your project as a .side file.

selenium save

The content of the file is simply xml, which is human readable. Here is the output from the simple test we setup.

You can see the first block covers the basic setup of the test within our project. We have the name of our project "name": "TestProject" along with the base url we specified "url": "https://www.google.co.uk" and the test name "name": "Test1".

  "id": "0e3a1d49-39f9-4eff-a1d8-f9ba98a2ea17",
  "version": "2.0",
  "name": "TestProject",
  "url": "https://www.google.co.uk",
  "tests": [{
    "id": "bb39e0cb-127b-4c78-b145-001123442d15",
    "name": "Test1",
    "commands": [{
      "id": "b3875247-77bf-4d9d-afbe-ff35a9ce81c0",
      "comment": "",
      "command": "open",
      "target": "/",
      "targets": [],
      "value": ""

The subsequent blocks of code in the xml in our .side file relate to each of our actions.

The first block sets the size of our web browser window to 814x607

{
      "id": "6ca1a67a-b4c0-490c-aaf0-2732331c38bb",
      "comment": "",
      "command": "setWindowSize",
      "target": "814x607",
      "targets": [],
      "value": ""
    }

We then select the input box and enter the value test

{
      "id": "bdd8b756-2049-4970-ae2d-c18e164db850",
      "comment": "",
      "command": "type",
      "target": "name=q",
      "targets": [
        ["name=q", "name"],
        ["css=.gLFyf", "css:finder"],
        ["xpath=//input[@name='q']", "xpath:attributes"],
        ["xpath=//form[@id='tsf']
	/div[2]/div/div/div/div[2]/input", "xpath:idRelative"],
        ["xpath=//div[2]/input", "xpath:position"]
      ],
      "value": "test"
    },

The entire process of recording a simple test is shown below.

selenium test001

Here is the full output of the .side file generated.

{
  "id": "0e3a1d49-39f9-4eff-a1d8-f9ba98a2ea17",
  "version": "2.0",
  "name": "TestProject",
  "url": "https://www.google.co.uk",
  "tests": [{
    "id": "bb39e0cb-127b-4c78-b145-001123442d15",
    "name": "Test1",
    "commands": [{
      "id": "b3875247-77bf-4d9d-afbe-ff35a9ce81c0",
      "comment": "",
      "command": "open",
      "target": "/",
      "targets": [],
      "value": ""
    }, {
      "id": "6ca1a67a-b4c0-490c-aaf0-2732331c38bb",
      "comment": "",
      "command": "setWindowSize",
      "target": "814x607",
      "targets": [],
      "value": ""
    }, {
      "id": "282d5900-77f1-4a39-9553-489af80b7f40",
      "comment": "",
      "command": "mouseOver",
      "target": "id=gbqfbb",
      "targets": [
        ["id=gbqfbb", "id"],
        ["css=#gbqfbb", "css:finder"],
        ["xpath=//input[@id='gbqfbb']", "xpath:attributes"],
        ["xpath=//form[@id='tsf']
	/div[2]/div/div[3]/center/input[2]", "xpath:idRelative"],
        ["xpath=//div[3]/center/input[2]", "xpath:position"]
      ],
      "value": ""
    }, {
      "id": "38ef3c8a-e69a-4dff-9ee4-3704824a241a",
      "comment": "",
      "command": "click",
      "target": "name=q",
      "targets": [
        ["name=q", "name"],
        ["css=.gLFyf", "css:finder"],
        ["xpath=//input[@name='q']", "xpath:attributes"],
        ["xpath=//form[@id='tsf']
	/div[2]/div/div/div/div[2]/input", "xpath:idRelative"],
        ["xpath=//div[2]/input", "xpath:position"]
      ],
      "value": ""
    }, {
      "id": "bdd8b756-2049-4970-ae2d-c18e164db850",
      "comment": "",
      "command": "type",
      "target": "name=q",
      "targets": [
        ["name=q", "name"],
        ["css=.gLFyf", "css:finder"],
        ["xpath=//input[@name='q']", "xpath:attributes"],
        ["xpath=//form[@id='tsf']
	/div[2]/div/div/div/div[2]/input", "xpath:idRelative"],
        ["xpath=//div[2]/input", "xpath:position"]
      ],
      "value": "test"
    }, {
      "id": "5ca7332f-466f-4fe7-80a5-e9e87068f511",
      "comment": "",
      "command": "click",
      "target": "css=#tsf > div:nth-child(2)",
      "targets": [
        ["css=#tsf > div:nth-child(2)", "css:finder"],
        ["xpath=//form[@id='tsf']/div[2]", "xpath:idRelative"],
        ["xpath=//form/div[2]", "xpath:position"]
      ],
      "value": ""
    }, {
      "id": "397b8fa7-274f-456e-a80e-cbe199a71fd3",
      "comment": "",
      "command": "click",
      "target": "css=center:nth-child(1) > .gNO89b",
      "targets": [
        ["css=center:nth-child(1) > .gNO89b", "css:finder"],
        ["xpath=(//input[@name='btnK'])[2]", "xpath:attributes"],
        ["xpath=//form[@id='tsf']
	/div[2]/div/div[3]/center/input", "xpath:idRelative"],
        ["xpath=//div[3]/center/input", "xpath:position"]
      ],
      "value": ""
    }]
  }],
  "suites": [{
    "id": "e25ec5e9-8715-4225-bcd1-92832ce0f3ab",
    "name": "Default Suite",
    "persistSession": false,
    "parallel": false,
    "timeout": 300,
    "tests": ["bb39e0cb-127b-4c78-b145-001123442d15"]
  }],
  "urls": ["https://www.google.co.uk/"],
  "plugins": []
}

Selenium Resources

https://github.com/SeleniumHQ

https://github.com/SeleniumHQ/docker-selenium

https://github.com/marketplace/actions/start-selenoid-server

Alternative: Katalon Studios

During this exploration of Selenium, I also tried looking into using Katalon Studios for automated UAT (User Acceptance Testing)

https://www.katalon.com/download/

https://github.com/katalon-studio/katalon-studio

https://docs.katalon.com/katalon-studio/docs/intro-RE.html#use-cases

I signed up sign up at the website. Downloaded and install the Katalon Studios application.

This application was large an over complicated for our project needs.

The entire sign up process was also very heavy with sales oriented questions during the setup, and subsequently, removal phase.

You may find this suits your needs, but for me, and this particular project, I am going to stick with Selenium IDE plugin for google chrome.

Presentation Slides

You can download the presentation slides here [PDF] AutomatedUserAcceptanceTests

You can read more about how I have used various techniques to improve testing and DevOps in my previous projects.

If you have any questions or comments you can contact me.

Creating your first programming language is easier than you think,
...also looks great on your resume/cv.