Noctam/DWP/ACS Test Harness

From The Dreaming
Jump to navigation Jump to search
ACS Test Harness

Meetings

Existing System

Specifications

Diary
2009:

Notes

Description

Noctam has been hired to produce a test harness for ACS, similar in scope to the FDS test harness that I wrote 2 years ago.

Current system

It's a replacement for what is currently there which is... not much. Basically:

  • The CISSTUB python server which mocks calls to CIS and NPS, running on local machine
  • The ACS application, running on local machine
  • A curl call to ACS to test it end-to-end.

There's no automation, no rainy-day-testing, not much in the way of statistics, etc. and it's extremely fiddly.

I did start a page on the existing system, but it doesn't go that much more in depth than this. It's a bit of a mess, but Mark Henkelis understands it all, and I'll be chatting with him about it on Friday (1st October), so I'll try and update the existing system section once I have that knowledge too.

Required system

The brief is to:

  • Containerise the components, so they all run in Docker; preferably in separate Docker images.
  • Replace the curl call with something more test-focused
  • Have some mechanism to start all servers necessary for the test(s)
  • Have the tests, inputs, outputs, any server configs, etc. be configurable
  • Make it entirely automatable, so a full suite can be run
  • Make it runnable via the CI server, so emitting meaningful test output, with stats, such that gitlab can report them at the front end.
  • Rewrite the CISSTUB application in java

Other things to consider

The CISSTUB server currently operates in multiple ways, namely:

  • Mock server for testing ACS
  • Logging proxy for accessing CIS
  • Mechanism for producing input files for later calls to CISSTUB

There is a push to replace it completely, so, assuming the ACS test harness and the logging proxy share code in some way (not a given), care will need to be taken to write this in such a way that it can be extended or part of it can be used as a library to implement the proxy functionality separately.

If the test definitions are going to be in JSON, we need to define the structure so that autocomplete and syntax checking can get rid of typos or omissions.

Initial implementation

There are 4 tests, written in a very basic fashion, using the current system.

The initial implementation should be able to run those tests, even if everything is not fully configurable yet. ie. A DCI908, DCI913, DCI923 and PDD test, with specific NINos testing that both ACS and CISSSTUB start up correctly, connections work correctly, data is passed around correctly, etc.

The implementation should work from, at most, three commands - "start servers", "run test" and "stop servers"; for the first draft, these can be separate calls so we don't have to deal with timing / readiness issues.

I want this to be in Docker from as soon as possible, but the first implementation probably won't be, for simplicity and getting something solid to look at as quickly as possible. That will be the very next thing to do once the first draft is working.

So this will need:

  • Certificate management
  • Server management
  • Initial data identification from inputs, expected outputs
  • Servers for CIS (possibly not for NPS; I'll have to check the DCI calls to see if these tests use it (they do))
  • (BASH) Scripts for starting, stopping servers.
  • Client java program for calling ACS
  • (BASH) Script for calling client java program.

I'm going to start planning the structure and deliverables a bit more first, so that once I get gitlab access, I can start creating the structure immediately and get going as soon as possible.

Preparation

Some things I'll need to set up:

  • Gitlab / Git
    • ...including access to fdacs and any other repos that I can identify later
  • JIRA
  • Confluence
    • Including where I should be documenting this stuff
  • IntelliJ, with some useful plugins:
    • PyCharm - so I can read all Mark's code a bit easier
    • JSON management - no idea what's available, I'll have a look around.
  • Python 3

First steps

I'm going to:

  • get my macbook set up
  • set up the current system and ensure I can run the tests as they stand
  • Write up some opening JIRA tickets to cover the initial implementation
  • go through the current (relevant bits of) CISSTUB code and understand them as much as I can
  • define an initial implementation structure - libraries, packages, scripts, inputs, outputs
    • maybe throw a quick diagram together to describe the server aspect, at least
  • write the proposed BASH script(s), using the current system
  • Document the initial implementation

After that

It will largely be led by that 'go through and understand as much as I can', I imagine, but, I will then...

  • Implement certificate handling
    • I will be trying to keep this separate-ish, so we can use this functionality elsewhere if required
  • Implement the basic runnings of the server(s), and the I/O to handle the 4 initial test cases
  • Tie up the server BASH scripts to use the new server
  • Make sure it continues to work with the new server, much like it did before.

And then

  • Do everything else
    • (I'll expand on this as I get closer to it)
Retrieved from "https://wiki.panglossian.net/wiki/index.php?title=Noctam/DWP/ACS_Test_Harness&oldid=338"