Mainframe-testaus – täydellinen opetusohjelma

Before learning Mainframe Testing concepts, let us first look at the platform the tests run on.

Mikä on Mainframe?

The mainframe is a high performance and high-speed computer system. It is used for large scale computing that requires high availability and strong security. It is mostly used in sectors like finance, insurance, retail and other critical areas where huge volumes of data are processed many times a day.

Mainframe testaus

Mainframe testaus is a process of testing software applications and services based on Mainframe Systems. The purpose of Mainframe Testing is to ensure the performance, reliability and quality of a software application or service through verification and validation methods, and to check whether it is ready to deploy.

While performing Mainframe Testing, the tester mainly needs to know the navigation of the CICS screens. These screens are custom built for specific applications. When changes are made to the code in COBOL, JCL and similar languages, the tester does not have to worry about the emulator set up on the machine, because changes that work on one terminal emulator will work on the others.

• The Mainframe application (otherwise called job batch) is tested against the test cases developed using requirements.
• Mainframe Testaus suoritetaan yleensä käyttöön otetulle koodille käyttämällä erilaisia ​​syötetiedostoon asetettuja tietoyhdistelmiä.
• Applications that run on the mainframe can be accessed through a terminal emulator. The emulator is the only software that needs to be installed on the client machine.

Because the platform behaves differently from a web stack, it helps to know which mainframe characteristics drive the test design. Mainframe Testing therefore sits alongside the other ohjelmistotestauksen tyypit sen sijaan, että korvaisi jonkun niistä.

Mainframe-attribuutit

1. Virtuaalinen tallennustila
   • Se on tekniikka, jonka avulla prosessori simuloi päämuistia, joka on suurempi kuin todellinen tallennustilan todellinen määrä.
   • Se on tekniikka käyttää muistia tehokkaasti erikokoisten tehtävien tallentamiseen ja suorittamiseen.
   • Se käyttää levytallennustilaa todellisen tallennustilan laajennuksena.
2. Moniohjelmointi
   • The computer executes more than one program at the same time. But at any given moment only one program can have control of the CPU.
   • Se on toiminto, joka on tarkoitettu CPU:n tehokkaaseen käyttöön.
3. Eräkäsittely
   • Se on tekniikka, jolla mikä tahansa tehtävä suoritetaan yksiköissä, jotka tunnetaan työtehtävinä.
   • Työ voi saada yhden tai useamman ohjelman suorittamaan peräkkäin.
   • Työn ajoittaja päättää, missä järjestyksessä työt tulee suorittaa. Keskimääräisen suorituskyvyn maksimoimiseksi työt ajoitetaan niiden prioriteetin ja luokan mukaan.
   • The necessary information for batch processing is provided through JCL (JOB CONTROL LANGUAGE). JCL describes the batch job — programs, data and resources needed.
4. Ajan jakaminen
   • Aikajakojärjestelmässä jokaisella käyttäjällä on pääsy järjestelmään päätelaitteen kautta. Sen sijaan, että käyttäjä lähettäisi työt, jotka on ajoitettu myöhempään suoritukseen, käyttäjä antaa komennot, jotka käsitellään välittömästi.
   • Tästä syystä tätä kutsutaan "vuorovaikutteiseksi käsittelyksi". Sen avulla käyttäjä voi olla suoraan vuorovaikutuksessa tietokoneen kanssa.
   • Aikaosuuskäsittely tunnetaan nimellä "Foreground Processing" ja eräkäsittely tunnetaan nimellä "Background Processing".
5. spooling
   • SPOOLing stands for Simultaneous Peripheral Operations Online.
   • SPOOL device is used to store the output of a program or application. The spooled output is directed to output devices like a printer (if needed).
   • Se on ominaisuus, joka hyödyntää puskuroinnin etua tulostuslaitteiden tehokkaaseen hyödyntämiseen.

Manuaalisen testauksen luokitus keskustietokoneessa

These attributes split manual test work on the mainframe into two clearly separated streams.

keskuskone Manuaalinen testaus voidaan luokitella kahteen tyyppiin:

1. Batch Job Testing —

• The testing process involves execution of batch jobs for the functionality implemented in the current release.
• The test results extracTulostetiedostoista ja tietokannasta saadut tiedot tarkistetaan ja tallennetaan.

2. Online Testing —

• Online Testing refers to testing of CICS screens, which is similar to testing a web page.
• Olemassa olevien näyttöjen toimintoja voitaisiin muuttaa tai uusia näyttöjä voitaisiin lisätä.
• Eri sovelluksissa voi olla kyselynäyttöjä ja päivitysnäyttöjä. Näiden näyttöjen toimivuus on tarkistettava osana online-testausta.

How to Do Mainframe Testing

1. The Business team prepares requirement documents, which determine how a particular item or process is going to be modified in the cycle of release.
2. The testing team and the development team receive the requirement document. They figure out how many processes will be affected by the change. Usually, in a release, only 20-25% of the application is affected directly by the customized requirement. The remaining 75-80% of the release effort goes to the out-of-the-box functionality, such as testing the surrounding applications and processes.
3. Mainframe-sovellus on siis testattava kahdessa osassa:
   • Testausvaatimukset — Testing the application for the functionality or the change mentioned in the requirement document.
   • Integraation testaus — Testing the whole process or other applications which receive or send data to the affected application. Regressiotestaus on tämän testaustoiminnan pääpaino.

Mainframe Automation Testing Tools

Alla on luettelo työkaluista, joita voidaan käyttää keskuskoneessa Automaatiotestaus.

• REXX — the scripting language shipped with z/OS, widely used to drive repetitive job submission and output checks.
• kunnostautua — used with macros to build, compare and report on test data and output files.
• OpenText UFT yksi — the current name of the tool the industry still calls QTP or QuickTest Professional; it automates 3270 terminal screens.
• Galasa — an open-source, Open Mainframe Project deep integration test framework that drives 3270 screens, JCL batch jobs and Db2 from a CI/CD pipeline.
• Vendor z/OS test suites - IBM Test Accelerator for Z and BMC AMI DevX Total Test cover COBOL unit testing and virtualised test environments.

Whichever tool is chosen, it only pays back when it sits inside a maintained test automation framework rather than a loose pile of scripts.

Mainframe-testauksen metodologia

Let us consider an example: An XYZ insurance company has a member enrollment module. It takes data both from the member enrollment screen and from offline enrollment. As discussed earlier, it takes two approaches for Mainframe Testing, online testing and batch testing.

• Online testing is done on the member enrollment screen. Just like a web page, the database is validated with data entered through the screens.
• Offline enrollment can be paper enrollment or enrollment on a third party website. The offline data (also referred to as batch) will be entered into the company database through batch jobs. An input flat file is prepared as per the prescribed data format and fed to the sequence of batch jobs. So for mainframe application testing we can use the following approach.
  • The first job in the line of batch jobs validates the data entered — for example special characters, or alphabets in number-only fields.
  • The second job validates the consistency of data based on business conditions. For example, a child enrollment should not contain dependent data, or a member zip code which is not available for service by the enrolled plan.
  • The third job modifies the data into the format that can be entered into the database. For instance, deleting the plan name (the database will store only the plan ID and insurance plan name), appending the date of entry, and similar changes.
  • Neljäs työ lataa tiedot tietokantaan.
• Batch job testing is done on this process in two phases —
  • Each job is validated separately, and
  • Integration between the jobs is validated by providing the input flat file to the first job and validating the database. (Intermediary results have to be validated for extra caution.)

The following is the method followed for Mainframe Testing:

Step 1) Shakedown/Smoke Testing

The main focus in this stage is to validate whether the code deployed is in the right test environment. It also ensures that there are no critical issues with the code. This is the mainframe equivalent of Savun testaus on any other platform.

Vaihe 2) Järjestelmän testaus

Alla on osana järjestelmätestausta tehtyjä testaustyyppejä.

1. Erätestaus — This testing is done by validating the test results on output files and the data changes made by the batch jobs under testing scope, and recording them.
2. Verkkotestaus — This testing is done on the front end of the mainframe application. Here the application is tested for correct entry fields like an insurance plan, interest on the plan, and similar values.
3. Online-eräintegraation testaus — This testing is done on systems having both batch processes and an online application. The data flow and interaction between the online screens and the batch jobs is validated.

   (Example for this type of testing — Consider an update on plan details such as an increase of interest rate. The change of interest is done on an update screen, and the balance details on the affected accounts will be modified only by a nightly batch job. Testing in this case is done by validating the plan details screen and the batch job run for updating all the accounts.)

4. Tietokannan testaus — The databases where the data from the mainframe application is held (IMS, IDMS, Db2, VSAM/ISAM, sequential datasets, GDGs) are validated for their layout and data storage.

Step 3) System Integraation testaus

Tämän testauksen ensisijaisena tarkoituksena on vahvistaa niiden järjestelmien toimivuus, jotka ovat vuorovaikutuksessa testattavan järjestelmän kanssa.

These systems are not directly affected by the requirements. However, they use data from the system under test. It is important to test the liitäntä and the different types of messages (like Job Successful, Job Failed, Database updated) that can flow between the systems, and the resulting actions taken by the individual systems.

Tässä vaiheessa tehtävät testityypit ovat

1. Erätestaus
2. Verkkotestaus
3. Online — Batch Integration Testing

Step 4) Regression Testing

Regression Testing is a common phase in any type of testing project. This testing in Mainframes ensures that batch jobs and online screens which do not directly interact with the system under test (or do not come in the scope of requirements) are not affected by the current project release.

In order to have effective regression testing, a particular set of test cases should be shortlisted depending on their complexity and a regression bed (test case repository) should be created. This set should be updated whenever new functionality is rolled out into the release. Where the regression bed is too large to run in full, Riskipohjainen testaus is used to decide which jobs and screens are re-run first.

Vaihe 5) Suorituskykytestaus

This testing is done to identify the bottlenecks in high hit areas like front end data entry and online database updates, and to project the scalability of the application. Long-running batch windows are usually examined with Stressitestaus against peak volumes.

Vaihe 6) Suojaustestaus

Tämä testaus tehdään sen arvioimiseksi, kuinka hyvin sovellus on suunniteltu ja kehitetty torjumaan tietoturvahyökkäyksiä.

Twofold security testing should be done on the system — mainframe security and network security.

The features which need to be tested are

1. Integrity
2. Luottamuksellisuus
3. Lupa
4. Authentication
5. Saatavuus:

Steps Involved in Batch Testing

1. After the QA team receives the approved package (the package contains procedures, JCL, control cards, modules and similar items), the tester should preview and retrieve the contents into PDS as required.
2. Convert the production JCL or development JCL into QA JCL, otherwise called JOB SETUP.
3. Copy the production file and prepare the test files.
4. For every functionality, there will be a job sequence defined (as explained in the example in the Methodology in Mainframe Testing section). The jobs should be submitted using the SUB command with the test data files.
5. Check the intermediate file in order to identify the reasons for missing or errored-out data.
6. Check the final output file, the database and the Spool to validate the test results.
7. Jos työ epäonnistuu, kelalla on syy työn epäonnistumiseen. Korjaa virhe ja lähetä työ uudelleen.

Testiraportointi - vika should be logged if the actual result deviates from the expected result.

Steps Involved in Online Testing

1. Select the Online screen in a Testiympäristössä.
2. Testaa jokaisesta kentästä hyväksyttävät tiedot.
3. Testaa Testiskenaario näytöllä.
4. Tarkista tietokanta tietojen päivityksiä varten online-näytöstä.

Testiraportointi — A defect should be logged if the actual result deviates from the expected result.

Steps Involved in Online — Batch Integration Testing

1. Run the job in a Test Environment and validate the data on the online screens.
2. Update the data on the online screens and validate whether the batch job runs correctly with the updated data.

Commands Used in Mainframe Testing

These steps are driven from the terminal, so a small command vocabulary covers most of a tester’s day.

1. LÄHETÄ — Submit a background job.
2. PERUUTA — Cancel a background job.
3. ALLOCATE — Allocate a dataset.
4. COPY — Copy a dataset.
5. NIMEÄ UUDELLEEN — Rename a dataset.
6. POISTA — Delete a dataset.
7. JOB SCAN — Bind the JCL with the program, libraries, files and other resources without executing it.

Tarvittaessa käytetään monia muita komentoja, mutta ne eivät ole niin usein.

Prerequisites to Start Mainframe Testing

Basic details needed for Mainframe Testing are:

• Kirjautumistunnus ja salasana sovellukseen kirjautumista varten.
• Brief knowledge of ISPF commands.
• Names of the files, the file qualifier and their types.

Before starting Mainframe Testing, the below aspects should be verified.

1. Job
   • Do a job scan (command — JOBSCAN) to check for errors before executing it.
   • The CLASS parameter should be pointed to the test class.
   • Direct the job output into spool or a JHS, or as required, by using the MSGCLASS parameter.
   • Reroute the email in the job to spool or to a test mail ID.
   • Comment out the FTP steps for initial testing and then point the job to a test server.
   • In case an IMR (Incident Management Record) is generated in the job, add the comment “TESTING PURPOSE” in the job or param card.
   • Kaikki työssä olevat tuotantokirjastot tulisi muuttaa ja osoittaa testikirjastoihin.
   • Työtä ei saa jättää vartioimatta.
   • To prevent the job from running in an infinite loop in case of any error, the TIME parameter should be added with a specified time.
   • Tallenna työn tulos, mukaan lukien kela. Kela voidaan tallentaa XDC:n avulla.
2. filee
   • Create a test file of the needed size only. Use GDGs (Generation Data Groups — files with the same name but with sequential version numbers, such as MYLIB.LIB.TEST.G0001V00 and MYLIB.LIB.TEST.G0002V00) when necessary to store data into consecutive files with the same name.
   • The DISP (Disposition — tells the system whether to keep or delete the dataset after normal or abnormal termination of the step or job) parameter for the files should be coded correctly.
   • Ensure that all the files used for job execution are saved and closed properly to prevent the job from going into HOLD.
   • While testing using GDGs, make sure that the right version is pointed at.
3. tietokanta
   • While executing the job or online program, ensure that unintended data is not inserted, updated or deleted.
   • Also, ensure that the correct Db2 region is used for testing.
4. Testata tapauksia
   • Always test for boundary conditions such as an empty file, first record processing and last record processing.
   • Sisällytä aina sekä positiiviset että negatiiviset testiolosuhteet.
   • In case standard procedures are used in the program, such as checkpoint restart, abend modules or control files, include Testitapauss to validate whether the modules have been used correctly.
5. Testitiedot
   • Testitietojen asetukset tulee tehdä ennen testauksen aloittamista.
   • Never modify the data on the test region without notifying others. There may be other teams working with the same data, and their tests would fail.
   • Mikäli tuotantotiedostoja tarvitaan suorituksen aikana, on hankittava asianmukainen valtuutus ennen niiden kopioimista tai käyttöä.

Esimerkkikäytäntöjä

1. In case of a batch job run, MAX CC 0 is an indicator that the job has run successfully. It does not mean that the functionality is working fine. The job will run successfully even when the output is empty or not as per expectation. So it is always expected to check all the outputs before declaring the job successful.
2. It is always a good practice to do a dry run of the job under test. A dry run is done with empty input files. This process should be followed for the jobs which are impacted by the changes made for the test cycle.
3. Before the test cycle begins, the test job set up should be done well in advance. This helps in finding out any JCL error in advance, hence saving time during execution.
4. While accessing Db2 tables through SPUFI (an option on the emulator to access Db2 tables), always set auto commit to “NO” in order to avoid accidental updates.
5. Test data availability is the primary challenge in batch testing. Required data should be created well in advance of the test cycle and should be checked for completeness. Tracking that preparation in a shared testien hallinta repository keeps the regression bed and the data setup aligned.
6. Some online transactions and batch jobs may write data into MQs (Message Queues) for transmitting data to other applications. If the data is not valid, it may disable or stop the MQs, and this will affect the whole testing process. It is good practice to check that MQs are working fine after testing.

Mainframe Testing Challenges and Troubleshooting

Even with those practices in place, a few problems recur on almost every mainframe release. The table below pairs each one with the approach that resolves it.

Haasteet	Lähestymistapa
Puutteelliset/epäselvät vaatimukset	There may be access to a user manual or training guide, but those are not the same as documented requirements. Testers should be involved in the ohjelmistotestauksen elinkaari from the requirements phase onwards. This helps to verify whether the requirements are testable.
Data Setup / Identification	There may be situations where existing data should be reused as per the requirement. It is sometimes difficult to identify the required data from the existing data. For data setup, homegrown tools can be used as per the need. For fetching existing data, queries should be built in advance. In case of any difficulty, a request can be placed with the data management team for creating or cloning the required data.
Työn asetukset	Once the jobs are retrieved into PDS, the job needs to be set up in the QA region, so that the jobs are not submitted with a production qualifier or path detail. Job setup tools should be used to overcome human errors made during setup.
Ad-hoc-pyyntö	There may be situations when alusta loppuun testaus needs to be supported due to a problem in upstream or downstream applications. These requests increase the time and effort in the execution cycle. Use of automation scripts, regression scripts and skeleton scripts can help in reducing the time and effort overhead.
Ajanmukaiset julkaisut laajuuden muutoksille	There may be a situation where the code impact completely changes the look and feel of the system. This may require a change to test cases, scripts and data. A scope change management process and impact analysis should be in place.

Common Abends Encountered

When a job fails, the spool reports an abend code. The list below covers the codes a mainframe tester meets most often, along with the usual cause.

1. S001 — An I/O error occurred.

   Reason — Reading at the end of the file, file length error, or an attempt to write into a read-only file.

2. S002 — Invalid I/O record.

   Reason — Attempt to write a record longer than the record length.

3. S004 — Error occurred during OPEN.

   Reason — Invalid DCB.

4. S013 — Error opening a dataset.

   Reason — PDS member does not exist, or the record length in the program does not match the actual record length.

5. S0C1 - Operation Exception.

   Reason — Unable to open file, or missing DD card.

6. S0C4 — Protection exception / storage violation.

   Reason — Trying to access storage not available to the program.

7. S0C7 — Program check exception, data.

   Reason — Change in record layout or file layout.

8. Sx22 — Job has been cancelled.

   Reason — The job was ended before completion; the middle digit identifies who or what cancelled it.

9. S222 — Job cancelled by the user without a dump.
10. S322 — Job or step time exceeded the specified limit, or the program is in a loop, or the TIME parameter is insufficient.
11. S522 — TSO session timeout.
12. S806 — Unable to link or load.

    Reason — The job is unable to find the specified load module.

13. S80A — Not enough virtual storage to satisfy GETMAIN or FREEMAIN requests.
14. S913 — Trying to access a dataset which the user is not authorized to use.
15. Sx37 — Unable to allocate enough storage to the dataset.

Error Assist — A very popular tool to get detailed information on various types of abends.

Common Issues Faced During Mainframe Testing

• Job Abends — For successful completion of the job, you should check the data, the input file and whether the modules are present at the specific location. Abends can occur for multiple reasons, the most common being invalid data, an incorrect input field, a date mismatch or environmental issues.
• Tulostiedosto tyhjä — Though the job might run successfully (MaxCC 0), the output might not be as expected. So before passing any test case, the tester has to make sure that the output is cross verified. Only then should testing proceed further.
• Syötetiedosto tyhjä — In some applications, files are received from upstream processes. Before using the received file for testing the current application, the data should be cross verified to avoid re-execution and rework.

UKK