Noctam/DWP/ACS Test Harness/Notes

From The Dreaming
Jump to navigation Jump to search
ACS Test Harness

Meetings

Existing System

Specifications

Diary
2009:

Notes

Notes

Just a scratchpad to make notes as I'm looking at stuff. Temp stuff - probably be wiped regularly.

CISSIM

At the moment, I'm ignoring a couple of things which I'll loop back to later and document - we may not need them for ACS testing, or they're pretty straightforward and we'll be implementing something brand new anyway. These things are:

  • 'NINO Lookup' - not 100% sure what this is/does. There's lots of config around it, but it looks fairly self-contained, so I'm comfortable looking into this in isolation.
  • 'Wait' timings - ultimately just a sleep(random()), with, er, predefined values of 'random'. The only complexity is in the configuration of the timing choices, really. I'll go through it properly later and get some requirements for what we need in terms of wait configuration for tests.
  • The config loading needs a further look - the search stuff, especially; it does quite a bit of analysis of the fields to search on... for PDD, if nothing else.
    • More on this - for the ACS Test Harness, it might not be so important, but for the stub/proxy, we will absolutely need some way of reaping the structural data for PDD screens - there's a lot going on here in order to make the configuration as straightforward as possible (which it is, believe it or not, compared to the alternative).

DCIxxx Handling

  • Reads config provided
  • 'FOLDER' is the folder in which the output files are stored
  • Handling for DCI calls puts together a lookup key from:
    • The 'dialog' - one of:
      • DCI908_R_RPLY
      • DCI913_R_RPLY
      • DCI923_R_RPLY
    • And the 'system id' from the ns1:systemID value of the provided XML.
  • It then uses those values to look up entries in the config for that type of call; the entry is a tuple made up of:
    • A list of search specs, each entry a tuple of made up of: 'searchKey', 'searchValue[;trimlength]'
    • output file
    • A list of replacements, each entry a tuple of 'output xml element name', 'replacement', where 'replacement' is typically @[input xml element name] - i.e. '@' (the value found at...) 'input xml element name' (the xml element with the provided name in the input).
  • It then just goes through through the replacements finding the output xml elements and replacing them with the values specified (reading them from the input XML document if (when) necessary)
  • Finally, it stringifies the XML document object it's been playing with, outputs it (interestingly, using chunked encoding) and goes about its business.

Some observations

OK, this is actually quite flexible, and, although the code is terse and largely uncommented, it's not that hard to follow. Ultimately, it doesn't do that much (in the above, quite narrow, use case) - I could fairly easily strip out everything that isn't required by a DCI call and just have that as our starting block of code... I might do that, though I want to get PDD and NPS in there too, neither of which are handled in my initial examination above.

PDD Handling

This differs to DCI, not least because the input is base64-encoded EBCDIC rather than an ASCII-based charset.

  • The EBCDIC charset used is cp037; the ASCII-based charset used is latin1 (aka ISO8859-1).
  • The first thing it does is base64decode the input; then it queries it to get some processing data:
    • Bytes 4-20: Message ID (henceforth 'txn type')
    • Bytes 70-86: (E1?) Process Name (henceforth 'dialog name')
      • Bytes 326-332: Overrides Process Name if Message ID is CIS1151RSTRTDIA and the above process name, trimmed, is "PD(375|380)" - the comment says: "for messages with multiple calls, [we] have to get the real call from the body"
    • Bytes 125-129: Um, "(E3B?) Infr TPSVCE Name" - as far as I can tell, basically the BASI. (henceforth 'service')
  • It then manipulates the 'txn type', extracting:
    • txn number from bytes 3-7
    • txn subtype at byte (offset) 7 - if E is found, it replaces it with U (?! No idea)
  • Combining the above to create the 'dialog' - the lookup to use in the config; e.g. PD350_7017_U_RPLY is made up of:
    • 'dialog name': PD350
    • 'txn number' (txn type[3-7]): 7017
    • 'txn subtype' (txn type[7]): U
    • 'direction' (hardcoded): RPLY
  • It uses the service and above compound dialog to look up the entry in the config, the above example would give:

SFRM, PD350_7017_U_RPLY, PD350.CIS7017UPRSNDETS.STUB.out (as far as I can tell, regardless of service - it's the same output file for each service).

  • The rest of the config provides:
    • The output file
    • The search parameters to identify which config entry to use (in theory - as far as I can tell only a single test entry uses this in PDD calls)
      • There is one entry that has a search parameter in the PVT config file, with an output filename of testonly - pretty sure this wouldn't work as written. It's written as {{{1}}} - the code appears to expect
    • The replacements, e.g. {{{1}}} - i.e. set H1151-FBK-NINO-GP to the value found in the input at W80-R1C-DAM-RQST-FN-KEY, trimmed to the first 9 characters.
      • The replacements are done using straight index / length replacements - mapping index/length to field name is handled in the config loading, which is going to need much further examination.
  • So the processing takes the above stuff from the config and processes it, identifying the correct file, and replacing as required. There's a lot of setting the run date, and setting a NINo, and not a lot else.

Further observations

Way more convoluted than DCI calls, because DCI calls are structured XML data using ASCII-based charsets. PDD is base64-encoded EBCDIC with the structure defined elsewhere, so just way more complicated right there.

That said it's still followable - the main complexity is in the config loading really. That's where it looks up the relevant field names and replaces them with (index, length) type tuples. The actual processing just uses them.

NPS Handling

So, after I got some more input, apparently the server script being used is pdservep.py. I've gone through that to try and get a handle on NPS - I still have no idea how it's called (i.e. what headers or content I should use), but I can break down the processing that occurs at least. It should be fairly simple, from what everyone seems to say... so...

  • To recognise an NPS call, the server looks at the path of the http request, specifically, it looks for paths.NPS_PATH which resolves to http://enirswebservices/JobcentrePlus/JobcentrePlus.asmx. So this is suspect in a number of ways:
    • Well, that's not a path - that's the entire URL; the other paths are just request paths, eg. /IB_CIS_VME_WEB/base64 or /cis_webservices_r2/services/CorporateCISDCI908NameTraceWS03.
    • Also, http and not https?
  • So, yeah - No idea what's going on with that.
  • For a recognised NPS path, it hardcodes the 'qtype' and 'dialog' to NPS_R_RQST and NPSR, respectively.
  • It validates the headers, erroring immediately if any unexpected values are found. It expects the following:
  • It validates that there is no whitespace in the SOAP message (?!)
  • It uses the above hardcoded qtype value (which, in the called method is called 'dialog' - not at all confusing) to return query and operatorid of "nino=x" and Nothing, respectively; the x in the former being the NINO from the submitted XML.
  • It creates a compound query from the query and dialog for the config file it's running with, something like: "NPSR;nino=AA111111A;"
  • After using the above query to find the response file, it compresses it (with zlib.MAX_WBITS|16 - i.e. including a gzip header in the output, apparently)
  • After that, it returns the output and exits. There is no replacement handling in this particular program, so it is a bit simpler.

Notes

So, this is the hackiest yet, really - probably the simplest, but there's an awful lot of hardcoding some stuff but not others, some ropey naming of local variables, destined to confuse (dialog meaning different things at different call levels). And I have no idea which config you would use with this and have no example input content.

Ultimately, this is all still a bit vague - I've basically overspecified what the code does (the above is practically code level) to try and make up for it. I'll have to come back to it really.

I do think this all could be done, fairly easily, with the same program rather than having two separate ones - it specifically uses the path of the HTTP call to determine whether to active 'NPS' handling over DCI (or other, I guess). I might try and combine them, though that would be risky without actually being able to test them properly.

Retrieved from "https://wiki.panglossian.net/wiki/index.php?title=Noctam/DWP/ACS_Test_Harness/Notes&oldid=309"