What is SOA Testing? Tutorial with Example

What is SOA Testing?

SOA (Service Oriented Architecture) Testing is the testing of the SOA architectural style, in which the application components are designed to communicate via communication protocols, typically over a network.

What is SOA?

SOA is a method of integrating business applications and processes together so as to meet the business needs.

In Software Engineering, SOA provides agility and flexibility to business processes. A change to a process or application can be directed at a particular component without affecting the whole system.

Software developers working in SOA either develop or buy chunks of programs called services.

What is Service?

The diagram below shows a payment gateway published as a service that several e-commerce sites can call.

• A service can be a functional unit of an application or a business process, which can be reused or repeated by any other application or process. (For example, in the image above, Payment Gateway is a service which can be reused by any e-commerce site. Whenever a payment needs to be made, the e-commerce site calls or requests the Payment Gateway service. After payment is completed on the gateway, a response is sent back to the e-commerce website.)
• Services are easy to assemble and easy to reconfigure components.
• Services can be compared to building blocks. They can construct any application needed, and adding or removing them from the application or business process is easy.
• Services are defined more by the business function they perform than as chunks of code.

Web Services

Most SOA services are exposed as web services, so the mechanics of a web service call are worth setting out before the test layers.

Web services are independent application components which are available over the web.

They can be published, found, and used on the web, and they communicate through the internet. The sequence below shows how a provider, a registry, and a consumer interact.

• The Service Provider publishes the service to the internet.
• The Client searches for a particular web service in the Web Service Registry.
• A URL and the WSDL for the required web service are returned. Using the WSDL and the URL, communication between the service provider and the requestor happens through SOAP messages.
• When a consumer calls a web service, an HTTP connection is established to the provider.
• A SOAP message is created to instruct the provider to invoke the required web service logic.
• The response received from the provider is a SOAP message which is embedded in the HTTP response. This HTTP response is the data format understandable by the consumer application.

Example

The screenshot below shows a weather report supplied by an external service and embedded in a search engine home page.

The home page of a website and a search engine display an everyday weather report. Instead of coding the weather report section from scratch, a weather report service can be bought from a vendor and integrated into the pages.

SOA Testing Layers

SOA consists of various technologies, and applications built using SOA have various services which are loosely coupled. The diagram below maps the three layers a test plan has to cover.

SOA Testing should focus on 3 system layers.

Services Layer

This layer consists of the services exposed by a system, derived from business functions.

For example, consider a Wellness Website which consists of:

• Weight Tracker
• Blood Sugar Tracker
• Blood Pressure Tracker

Trackers display the respective data and the date it was entered. The services layer consists of the services which get the respective data from the database:

• Weight Tracker service
• Blood Sugar Tracker service
• Blood Pressure Tracker service
• Login Service

Process Layer

The process layer consists of the processes, the collection of services which are part of a single functionality.

The processes might be part of a user interface (for example, a search engine) or part of an ETL tool that pulls data from the database.

The main focus in this layer is on user interfaces and processes. The user interface of the weight tracker and its integration with the database is the primary focus.

The following functions are of consideration:

• Adding new data
• Editing existing data
• Creating a new tracker
• Deleting data

Consumer Layer

This layer mainly comprises user interfaces, as the screen below illustrates.

Based on these layers, the testing of an SOA application is distributed into three levels:

• Service level
• Interface level
• End to End level

The two directions of travel differ: a top-down approach is used for test design, while a bottom-up approach is used for test execution.

Strategy for SOA Testing

Test Planning Approach

• The complete architecture of the application should be understood by the SOA testers.
• The application needs to be broken down into independent services (a service that has its own request and response structure and does not depend on any other service to form a response).
• The application structure needs to be reorganised into three components — data, services, and front-end applications.
• All the components need to be carefully analysed, and business scenarios should be chalked out.
• The business scenarios should be classified as common scenarios and application-specific scenarios.
• A Traceability Matrix should be prepared, and all test cases should be traced to business scenarios.

Test Execution Approach

• Each service component should be tested.
• Integration Testing of the service components should be done to validate the data flow through the services and data integrity.
• System Testing of the complete model should be done to validate the data flow between the front-end application and the database.
• Performance Testing should be done for fine tuning and optimum performance.

SOA Testing Methods

1) Business scenario driven data based testing

• Various business aspects related to the system should be analysed.
• Scenarios should be developed based on the integration of the various web services of the application, and of web services with the application.
• Data set-up should be done based on the above scenarios.
• Data set-up should also cover end-to-end scenarios.

2) Stubs

• Dummy interfaces are created to test services.
• Various inputs can be provided through these interfaces, and the outputs can be validated.
• When an application uses an interface to an external service which is not under test (a third-party service), a stub can be created during Integration Testing.

3) Regression testing

• Regression Testing on the application should be done when there are multiple releases, so as to ensure the stability and availability of the systems.
• A comprehensive regression test suite will be created covering the services which form an important part of the application.
• This test suite can be reused across multiple releases of the project.

4) Service Level Testing

Service level testing includes testing the component for functionality, security, performance, and interoperability. Each and every service needs to be tested independently first.

5) Functional Testing

Functional Testing should be done on each service to:

• Ensure that the service delivers the right response to each request.
• Ensure the right errors are received for requests with invalid or bad data.
• Check each request and response for every operation the service has to perform at run time.
• Validate the fault messages when an error occurs at the server, client, or network level.
• Validate that the responses received are in the right format.
• Validate that the data received in the response corresponds to the data requested.

6) Security Testing

Security testing of the web service is an important aspect during service level testing of the SOA application, because it ensures the safety of the application.

The following factors need to be covered during testing:

• The industry standard defined by WS-Security should be abided by the web service.
• Security measures should work flawlessly.
• Encryption of data and digital signatures on the documents.
• Authentication and authorisation.
• SQL injection, malware, XSS, CSRF, and other vulnerabilities are to be tested on the XML.
• Denial of service attacks.

7) Performance Testing

Performance Testing of the service needs to be done because services are reusable and multiple applications might be using the same service.

The following factors are considered during testing:

• Performance and functionality of the service need to be tested under heavy load.
• The performance of the service needs to be compared when it works individually and when it is coupled within the application.
• Load testing of the service should be performed to verify response time, to check for bottlenecks, to verify the utilisation of CPU and memory, and to predict scalability.

8) Integration level testing

• Service level testing ensures the proper working of the services individually; it does not guarantee the working of the coupled components.
• Integration Testing is done focusing mainly on the interfaces.
• This phase covers all the possible business scenarios.
• Non-functional testing of the application should be done once more in this phase. Security, compliance, and Performance Testing ensure the availability and stability of the system in all aspects.
• The communication and network protocols should be tested to validate the consistency of the data communication between the services.

9) End to End testing

This phase ensures that the application conforms to the business requirements both functionally and non-functionally.

The items below are ensured to be tested during end-to-end testing:

• All services working as expected after integration
• Exception handling
• User interface of the application
• Proper data flow through all the components
• Business process

Challenges in SOA Testing

Applying those methods is rarely straightforward, and the difficulties below recur on almost every SOA programme.

• Lack of interfaces for services.
• The testing process spans multiple systems, which creates complex data needs.
• The application is a collection of various components which tend to change, so the need for Regression Testing is more frequent.
• Due to the multilayer architecture, it is difficult to isolate defects.
• Since a service is used by different interfaces, load is difficult to predict, which makes performance test planning cumbersome.
• SOA is a collection of heterogeneous technologies. Testing an SOA application requires people with different skill sets, which in turn increases planning and execution costs.
• Since the application integrates multiple services, security testing has its own share of woes. Validating authentication and authorisation is difficult.

SOA Testing Tools

There are many SOA testing tools available in the market to help testers in testing SOA applications. Here are some of the popular SOA testing tools.

1) SoapUI

SoapUI is an open-source functional testing tool for services and API testing.

• Desktop application
• Supports multiple protocols — SOAP, REST, HTTP, JMS, AMF, JDBC
• Web services can be developed, inspected, and invoked.
• Can also be used for load testing, Automation Testing, and security testing
• Stubs can be created by MockServices
• Web service requests and tests can be generated automatically through its web service client.
• Has inbuilt reporting tools
• Developed by SmartBear, which ships both the open-source SoapUI distribution and the commercial ReadyAPI edition

2) Broadcom Service Virtualization (formerly iTKO LISA)

LISA is a product suite which provides a functional testing solution for distributed systems such as SOA. The product passed from iTKO to CA Technologies and is sold today as Broadcom Service Virtualization.

• Can also be used for regression, integration, load, and Performance Testing.
• Can be used to design and execute tests.

3) UFT One (formerly HP Service Test)

Service Test is a functional testing tool which supports both UI and shared services testing. Its API testing capability was folded into Unified Functional Testing, now sold by OpenText as UFT One.

• Both functional and performance tests of services can be done by a single script.
• Integrated with Quality Center, now sold as OpenText ALM / Quality Center.
• A massive amount of service and data can be managed.
• Supports interoperability testing by simulating JEE, AXIS, and DotNet client environments.

4) Parasoft SOAtest

Parasoft SOAtest is a testing and analysis tool suite developed for API and API-driven application testing.

• Supports web services, REST, JSON, MQ, JMS, TIBCO, HTTP, and XML technologies.
• Functional, unit, integration, regression, security, interoperability, compliance, and Performance Testing are possible.
• Stubs can be created using Parasoft Virtualize, which are more capable than SoapUI MockServices.

SOA Testing Use Cases

The worked example below applies the strategy, methods, and tools above to a single e-commerce site, phase by phase.

Consider an e-commerce website which contains the functions and sub-functions below.

Order Processing

The chart below breaks order processing into the sub-functions that become services.

PHASE 1

In the first phase of SOA testing, the test strategy phase, the application is broken into services and business functions.

Let us consider the services below in the application.

• Create Order
• Check Customer Status
• Change Order Status
• Check Order Status
• Check Inventory

Business functions are the same as the functions of the website.

Note: The test strategy document would contain the list of the services and the functions which have to be tested.

PHASE 2

This is the test planning phase. Test cases are written for each level.

End to End level. The test cases are written for each business use case and flow. Below are examples of test cases.

• Create an order with an active user.
• Create an order with an inactive user.
• Create an order with an available product with order quantity < available quantity.
• Create an order with an available product with order quantity > available quantity.
• Create an order with multiple items.
• Cancel an order completely.
• Cancel an order partially.

Integration level. Test cases are written for the integration of database and user interface. Below are example test cases.

• Create a new order with a single item. Verify that the order is created on the database.
• Create a new order with a single item. Verify that the price calculated for the order is correct.
• Create a new order with a single item. Verify that the quantity of the available product is reduced by the order amount.
• Verify that the status of the order displayed on the UI is the same as that on the database.
• Cancel the order and verify that the status of the order is modified on the database.
• For a first-time payment, verify that the payment details entered on the UI are saved on the database.
• For returning payments, verify that the payment details on the database are displayed on the UI.

Service level. Each service is tested for all the data conditions. Below are a few examples.

No.	Order Details	Order Condition
1	Create Order. No. of Items = 1	Quantity on order < Quantity on database
2	Create Order. No. of Items > 1	Quantity on Order < Quantity on database
3	Create Order. No. of Items = 1	Quantity on Order > Quantity on database
4	Check Order status	Status on database = Active
5	Check Order status	Status on database = Shipped
6	Check Order status	Status on database = Cancelled
7	Check Order status	Order id = Invalid
8	Check product availability	Quantity of product >0
9	Check product availability	Quantity of product =0
10	Check product availability	Product id = invalid

PHASE 3 — Test Execution

Test execution uses a bottom-up approach: service level testing is done first, then integration level, and at last end-to-end testing.

1) Service level

Let us consider that the SoapUI tool is used for testing the application. The WSDL and URL are browsed into the test window of SoapUI, and the request for each service is displayed in the request window. By modifying the data as per the service level test cases, requests are created for each test case.

Test Case	Request	Expected response
Create Order. No. of Items = 1, Quantity on order < Quantity on db	<productid>x2</productid><quantity>2</quantity>	<orderid>o3251</orderid><message>Successful</message>
Create Order. No. of Items > 1, Quantity on Order < Quantity on db	<productid>y1</productid><quantity>1</quantity><productid>y2</productid><quantity>3</quantity>	<orderid>o3251</orderid><message>Successful</message>
Create Order. No. of Items = 1, Quantity on Order > Quantity on db	<productid>x23</productid><quantity>200</quantity>	<orderid>null</orderid><message>Unsuccessful</message>
Check Order status. Status on database = Active	<orderid>o9876</orderid>	<status>Active</status><message>Successful</message>
Check Order status. Status on database = Shipped	<orderid>o9656</orderid>	<status>Shipped</status><message>Successful</message>
Check Order status. Order id = Invalid	<orderid>y5686</orderid>	<status>null</status><message>Unsuccessful</message>
Check product availability. Quantity of product >0	<productid>d34</productid>	<quantity>34</quantity><available>yes</available><message>Successful</message>
Check product availability. Quantity of product =0	<productid>y34</productid>	<quantity>0</quantity><available>no</available><message>Successful</message>
Check product availability. Product id = invalid	<productid>sder</productid>	<message>Unsuccessful</message>

2) Integration Level

The integration level test cases are executed on the user interface and database. Create an order with a single item:

• A user opens the website.
• The user goes to place an order.
• The user selects a valid product and quantity and saves the order.
• A message saying the order is placed successfully should be displayed.
• The user opens the database and checks whether the details of the order are the same as those entered on the website.

3) End to End level

The business flows and use cases are executed on the user interface. Create an order with multiple items:

• A user opens the website.
• The user goes to place an order.
• The user enquires about a valid product and quantity and adds them to the cart.
• Other valid products are added with valid quantities and the order is saved. Payment is made through a new payment method and the order is placed.
• A message saying “Order placed successfully” should be displayed.
• A tester should validate that the whole flow completes without skewing of data.

FAQs