LoginRadius 'Idx-Auto-Tester' Open Source Automation

About:

Idx-Auto-Tester is LoginRadius Identity Experience Automation Framework, which refers to IEF Automation. It is an open-source automation framework built-in Nightwatch| Node.js tool that delivers the standard authentication cases of LoginRadius Identity Experience.

Automation scripts are written in the Nightwatch framework.
Node.js Core Assertion Testing Library is used for assertions.

Release Roadmap:

The full-version releases that include more test coverage and major changes with several improvements & code optimizations will be coming up in the multiple scheduled major releases. The details can be found here.

Changelog:

All notable changes can be found here

Contribution:

The idx-auto-tester is open for the community to contribute. Read the contribution guidelines here.

Getting Started:

The Standard Authentication functionality, which is available with LoginRadius Identity Experience Framework, can be tested using these automated scripts. By running these automation scripts, you can ensure your implementation. The automation scripts will be continuously improved by adding more test cases whenever any new functionality is introduced in the LoginRadius Identity Experience Framework.

Usage:

Clone this repo to your local machine

Configure:

Rename fileconfig\config.temp.js by config\config.js

├── config
│   ├── inputs
│   ├── config.temp.js
├── helpers
├── test
├── package.json
├── nightwatch.js
└── nightwatch.json

Add your apiKey, apiSecret, and siteName in config\config.js

Don't have Loginradius App ?. Create your own here

Add mail7 apiKey and apiSecret for disposable email addresses
Mail7.io is an open-source service, mail7 apikey and secret can be retrieved by signing up at mail7.io
Open Terminal in IDE like VS Code or Sublime Text
Add Project dependencies by command npm install
You also can install dependencies by running command for a particular package like npm install -g nightwatch

Running Tests:

Now you are able to run scripts on your implementation of Loginradius Identity Experience.

Command to run all tests at once node nightwatch.js test
Command to run a single file node nightwatch.js test\<filename>.js
If you have installed Nightwatch globally (with -g option), the binary nightwatch will be available anywhere and test can be run by directly using nightwatch as a test runner nightwatch test\<filename.js>
An executable Batch file is also added with named as executable.bat which can be run with a double-click, and written command will be executed and HTML output report will be generated.

Nightwatch includes a command-line test runner, which makes it easy to run tests and generate useful output. Please refer to the Installation section for details on how to get the runner installed. There are a few different options on how to use the test runner, depending on your installation type.

Parallel Running:

The Test Case can also be run in parallel, as the test files will be triggered at once. Each test file will fill a test worker slot. Individual tests/steps in a test file will not run concurrently.

Open nightwatch.json file and look for below code

"test_workers": {
    "enabled": false,
    "workers": "auto"
}

The workers option configures how many child processes can run concurrently.
"auto" - determined by number of CPUs e.g. 4 CPUs means 4 workers
{number} - specifies an exact number of workers

The more details on parallel run can be found here.

Configure Headless Mode:

Search for --headless-none in nightwatch.json and replace it by headless like below code:

"chromeOptions": {
    "args": [
        "headless" ]
}

A headless browser is a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command-line interface or using network communication.

Limitations:

Currently, Test Cases are scripted to run with only Required Email Verification Flow.

Implementation:

The process of automation framework implementation or creation is defined here. You can refer to it for creating and updating the test cases with different scenarios.

Reporting:

XML Reporting:

By default, tests will generate a JUnit formatted XML report for each test file in the tests_output folder at the root of the project.

HTML Reporting:

Create a folderhtml-reportsat project root.
You can use interactive html-report also simply by running node nightwatch.js --reporter html-reporter.js test command.
HTML report will be saved as html-reports\siteName-datetime.html

If you'd like to define your reporter in addition to the built-in ones, you can follow the detailed guide here.

Know more about its features and benefits:

1. Tested Automation Scripts

With ready and tested automated scripts, you can quickly test the authentication features and workflows implemented using the LoginRadius Identity Experience Framework.

It covers following features and benefits:

Shorter execution time for Validation
As it is an automation script covering all the positive/negative test cases, the validation of the implementation is faster. Hence lowers administrative cost.
Minimal Configuration needed
It automates the entire test suite quickly with minimal configuration and is readable as well as easy to update.
Scenario-based Test cases
Each test case is covered in a way that every scenario is captured end to end to understand the consumer’s journey right from registration, login, forget the password, profile editor etc.
Error and Exception Handling
Error and exceptions handling is supported for failed assertions.

2. Support for Manual Test Cases

Mining the test cases by reviewing code is difficult, so we have extended support to deliver the written Test Cases with the expected outcomes as well.

3. Robust Framework

We have used Nightwatch.js end-to-end testing framework to create the Idx-Auto-Tester tool, which comes with the following benefits:

Robust automation framework
The capability of multi-browser compatibility runs the tests either sequentially or in parallel, together, by group, tags, or by single, that proves to be time-efficient.
Resolve the asynchronous queuing system problem
We are using 'Function-style commands' of Nightwatch to create commands while running the test cases.
Continuous Integration
It internally uses the powerful W3C Webdriver API or the Selenium Webdriver and simplifies writing end-to-end automated tests in Node.js and effortlessly sets up for integration.
Scalability
The automation script is scalable enough to incorporate any additions in the feature list, so in case there are new changes or enhancements, those will be easily taken into considerations.

4. Test Results and Reporting

The test results can directly be read from the terminal and also stored at a specified output folder.

Reports can be defined in supporting means-:

XML report for each test file
Interactive HTML report
Also, you can define reports in addition to the built-in ones.
Graphical Representations of HTML reports are planned to be accomplished as per roadmap.

5. Easy to read Documentation

With very effective documentation (readme, descriptive comments to test cases, commands) it is very easy to use by even a non-technical person.

6. Open Source

The tool is open for the community to contribute with the pull request process and standards defined by LoginRadius here.