GIZ-Nord » research and projects » RAS-CHEM 2.0.0 Test Report
 

RAS-CHEM 2.0.0 Test Report

  • Drucken

Alerting System for Health Threats Phase II (ASHT II)

Technical Evaluation of RASCHEM Database System version 2.0.0 RC Patch 0 build 24/11/2009, accessed via World Wide Web

Andreas Schaper, Herbert Desel, GIZ-Nord Poisons Centre Göttingen

Testing started: 2009-12-02
Last access: 2010-01-25 (all additions and alterations after 2nd export  of draft document for formatting and importing it to CIRCA on 2010-01-05 - are indicated by bold text style)

This report summarizes test results of the authors and test results by other project members that were discussed at the ASHT II Interim Meeting in Prague, January 2010.

Test Tools / Methods / Access to the System

Evaluation was performed using Mozilla Firefox 2.0.0.11 (running on Mac OC X) for most tests. Other versions of this Mozilla Firefox and two versions of MS Internet Explorer were used for comparison of several aspects of graphical layout of the programme.

The system was accessed via WWW address:

-    https://webgate.acceptance.ec.europa.eu/ras-chem

Obviously, the access is redirected to "The Cas server of DG Sanco" (SanCas) for login precedure.
- the browser transiently displays the title line "Yale Central Authentication Service" (which occationally could lead to doubt correct access by some users)

Testing was mainly performed via a Vodafone DSL network in Germany.

This testing is in part based on the Evaluation report of RAS CHEM version 1.1.2 build 17/03/2006 by Andreas Schaper and Herbert Desel, GIZ-Nord Poisons Centre Göttingen of 2007-02-12 (see ref. 2).

System login
  • Login was denied several times. Finally login was possible for user „deselhe" with changed password supplied by email.
  • For the user "schapan" (Andreas Schaper) it worked the same way, but finally it worked without problems
  • Once logged in the user was only logged out automatically after more than 30 min without activity (frequent auto logout was observed with in the previous RAS CHEM version)
  • The system responded fast within 1-4 s on all occations during the test
General Comments

During the testing an event created by another author was changed by mistake.

  • Restrictions or at least a warning should be displayed when somebody edits another author's event.

In the history subsection o each event, the authors that have edited the event should be indicated if more than one user would be allowed to change the documents.

Mouse actions were needed to use the system during testing.

  • Browsing and editing (all actions) should also be possible without mouse actions.
Screen labeled "List of Events"

After a successful user login to the system key data of the (latest) events are
displayed in temporal order ("List of Events", 6 columns, 1-5 screen lines per
message). The newest event is displayed on top of the list (as scheduled).

In the beginning of the testing 3 recently created events are displayed.

On the right hand side of the browser window, there is an error  in display of a programme menue: The "menu line" Event - Meetings - References is displayed to low hiding the button "Create Event".

Overlay of menu / buttons makes it difficult to create an event

  • this problem does not occur if a newer version of Firefox browser is used on another system

In column 1, the unique case ID is presented. This ID consists of a country code followed by a date

  • events created by some testers (all from UK) no country code is displayed.
  • For all users the country code should be displayed in the unique case ID field.

In column 3 the clinical effects and location are displayed (this makes entry rather long and this is not the intended content).

  • the clinical summary should be displayed instead

- location should be displayed in column 4

Further testing:

-    There is one event with more than one clinical symptom

The technical columns 1 to 3 counted from the right hand side need almost half of the table window for little information content. These columns should be made smaller.

Filter functions work well for all items tested.

Keyword search seems to work only in field clinical summary.
-    search should be extended to all free text fields.

After performing a keyword search it is not obvious how the complete list can be displayed again (especially necessary after unsuccessful search).

  • a "Restore list" button should be added here (also useful after selecting filters and after displaying Event Details, see below)
Screen Labeled "Event Detail"

By clicking on the "Unique Event ID" all details of the chosen event could be displayed.

Event detail screen opens, and complete event data is displayed

The data presented are organized in several subsections as scheduled.

  • the first subsection was labelled "Event, it should be labeled "Event Control Data"
    • Message type (chosen for compatibility with EWRS) is not displayed
    • "Alert level" is labeled "Level" (content displayed as scheduled, including definitons)
    • Date and time is displayed as scheduled
    • Reporting organisation is missing
    • Source of information is included as scheduled
  • The subsection "Data on Toxic Agents" allow more than one key chemical
    • this was not scheduled. It has advantages and disadvantages to be discussed in the near future
    • the chemical/agent groups are displayed as scheduled
    • the vehicle category parameter list is displayed as scheduled
  • The subsection "Exposure Data" is displayed as scheduled
    • the parameter list for circumstances of exposure are displayed correctly
  • The subsection "Clinical case description" is displayed as scheduled
  • A subsection "history" was added although not scheduled before.
  • The subsection "Comment" is displayed correctly
    • attachments are listed only when this subsection is 'opened'
    • therefore the last subsection should be named 'Comment and Attachments'
  • Content of longer free text fields, e.g. „Exposure Data Details" is displayed in small text window and has to be browsed to read.
    • all text should be displayed without window frames to facilitate easy reading

Printing of the whole dataset of an event should be made possible

  • not possible in this version

The history section was not scheduled in detail, but it looks very interesting.

Screen Labeled "Create Event" / "Edit Event"

General Points:

  • Create event screen opens, data fields can be edited
  • All fields are equipped with a "?"-button. By moving the mouse pointer over these buttons help texts are displayed after a short delay. Nothing happens when the "?"-button is clicked.
    • The help text should also show up when the "?"-button is clicked.
  • The help text disappears after 5 seconds. Long text messages cannot be read to the end without multiple repetitions of mouse movement
    • Help texts should be displayed as long as the user needs it, e.g. until mouse pointer is moved out of the button area.
    • (Comprehensive testing of the help buttons text will be carried out when prolonged display is possible.)
  • for the user it is not clearly visualized which fields are mandatory and which fields are voluntary
    • some field name descriptions (mandatory fields as date and time ..., clinical summary as well as voluntary fields as key chemical) are coloured in red. The reason for this is not clear for the testers
      • different colouring should be explained
    • field description for mandatory fields should be labeled with asteriks (*) - as in many other Web forms. In addition, descriptions for mandatory fields should be displayed more prominently (e.g. bold text style, volunary fields normal text style)
      • this should be done for parameter list based fields as well as for free text field (clinical summary)
  • All parameter list are populated as scheduled
  • Parameter lists are ordered in alphabetic order. This means that "special options" as "other(s)" or "unknown" often appear in the middle of the list.
    • special option should be displayed at the bottom of the lists
  • default values for fields connected to parameter list (Alert Level, Source of Information, Chemical/Agent Group, and  with special interest "Clinical Effects") should be provided as "not chosen yet" when starting the creation of new event - this has lead to many input errors during the testing.
    • when saving the event a warning should be displayed if one of the parameter list connected fields is (still) connected to "not chosen yet"
  • empty lines should be deleted in all default texts of all free text fields (see detailed description for field "Case Details" in section "CLINICAL CASE DESCRIPTION"
  • Sudden end of data input if data entry in any free text field is finished by carriage return key - all input is lost. If the mouse action is used to move the cursor to another field or to finish editing this problem des not occur.
    • this problem hampers testing and will hamper any future use of the system and therefore should be changed


Comments on Specific Fields:

EVENT CONTROL DATA

  • Field "Message Type" comes with a parameter list of 3 entries (Very Long Event, Miscellaneous Event, Event with no information)
    • EWRS message types should be used instead:  (1) "important information without exposure", (2) "enquiry without exposure", (3) "event with exposure"
  • With new addition of field "Message Type" in the lateast version of the RAS-CHEM description the Alert Level 1 (background information only) became redundant and should
    • either be set automatically, i.e. Alert Level 1 for Message Type 1 and 2 and Alert Level 2,3,4 or 5 for Message Type 3,
    • or Alert Level 1 should be disabled for Message Type 3 and Alert Levels must not be chosen for Message Types 1 and 2.
  • The help text related to the field Alert Level contains identical  text as the parameter list.
    • The help text for the Alert Level should be changed to 'Choose alert level to indicate the relevance of the event. If alert level 3 is selected an automatic email message will be sent to all registered system users. If alert level 1 or 2 is selected an automatic sms text message will be sent in addition to all users'
  • Field "reporting organisation" is labeled, but content is not displayed (the field should be filled automatically with the organisational data of the user)
  • Field "clinical summary" is restricted to about 50 characters (as scheduled).
    • The displayed field size should be adapted to this limitation (or get a message indicating this restriction)
  • Selection of date and time of outbreak works as scheduled
    • But: during the testing it was realised that in many cases date and time is not known precisely, at least in the beginning of the reporting.
    • Therefore, the format should be changed in a way that only part of the information can be included (e. g. only date but not time, or only month and year)
    • The year should be mandatory, all other fields specifiing data and time should be voluntary
    • The help button text should be: 'Enter the data and time of onset of the event. Leave corresponsing fields blank if time or day of month is unknown.'
  • "Scientific literature (see section 'event control')" should be added to the parameter list for "Source of Information"
  • The field "Further Data on Event" should be renamed "Further technical data/References" - as during testing many users wrongly include clinical data here.
    • The corresponding '?'-button text:  'Specific the source(s) of information for this event in detail. Give event / case ID, responsible person, other authorities/institutions involved. Do  n o t  include information on clinical course here'
  • Reason for reporting (taken from RAS-BICHAT) is presented as a parameter list from which only one item can be chosen
    • instead reasons for reporting should become radio buttons - this means more than one can be chosen
  • 31a - Public health impact serious? 31b - Event unusual or unexpected? 31c - Significant risk for international spread? 31d - Significant scope for international coordination/cooperation?
    31e - Event relevant for health security activities?   

DATA ON TOXIC AGENT(S)

  • Comment on long (parameter) list of Key Chemicals included in RAS CHEM see below

  • The RAS-CHEM concept allows no or one key chemical for every event (voluntary field).

    • testing has revealed that more than one key chemical can be chosen
    • it has to be discussed and evaluated during further testing whether this fits to the system's objectives

CLINICAL CASE DESCRIPTION

  • "Clinical Summary" should be a mandatory free text field. In the version tested it is possible to save ("submit") an event with empty field  "Clinical Summary" (create or edit)
    • Submission of an event should only be allowed if the field "Clinical Summary" contains texts
  • for the (non mandatory) field "Clinical effects" additional entries to the parameter list shall be 
    • 'no symptoms observed (yet)'
    • 'other symptoms (see case description)'
    • furthermore: during testing it became obvious, that the list on clinical effects is much to short to describe all events sufficiently
      • a list with about 1000 symptoms has been developed within the project and will be provided for inclusion
  • Field Case Details: standard text that is display in text windows on creation of a new event should contain no empty lines between the entries,i.e.
o    Case Details:
o    Poisoning Severity:
o    Diagnostic results (Laboratory):
o    Therapy:
o    Outcome:"

 

instead of "
o    Case Details:
o    
o    
o    Poisoning Severity:
o    
o    
o    Diagnostic results (Laboratory):
o    
o    
o    Therapy:
o    
o    
o    Outcome:
o    
o    "

ATTACHMENTS

  • loading attachments: .rtf is no allowed
    • .rtf should become an allowed format of attachment (rtf is free of virusses - in contract to .doc)



Input of key chemical:

Choice of a key chemical might (at least in some cases) will be important for the identification of similar or even identical cases reported to the System.

Therefore the correct choice of the key chemical is an important part of RAS-CHEM's function and power.

At a meeting in Luxembourg in spring 2009 between the ASHT project members and the programmers it was decided to use a (long) parameter list to assist the user when selecting the key chemical. This seemed to be necessary to avoid unrecognized double entries of the same agents when using synonyms (and not recognized that they are synonyms) and to avoid misspellings.

It was decided to use the agent list which was created during the REACH preregistration process, expanded by agents of special terroristic concern (combined list is named 'REACH ASHT Table of Chemicals', RATOC). A version of this list (named RATOC I), downloaded from the ECHA website and expanded by the ASHT team based on the agent of concern identification in ASHT Phase I was made available to the programming team (EXCEL file with 4 tables included, access on GIZ-Nord Website: here).

  • testing reveals that only a subset of RATOC I was used as key chemical parameter list for RAS-CHEM V 2.0.0.0
  • convenient selection of agents is facilitated by displaying a selection of names (that contain the text string typed) below the input line. The  selection displayed is updated when another key is pressed
  • important agents that are included in RATOC I (table 4), e. g. sarin and methanol (two agents causing historic mass intoxications), cannot be found in the list
  • instead of a subset, the whole RATOC I list should be used as parameter list of selecting the key chemical within the system

 

  • testing reveals that is possible to create additional key chemicals that are not included in the paramater list
  • instead, this should be disabled to avoid the problems in agent description describes above.

 

  • a substantial add-on of substances/compounds of special toxicological concern (e. g. nerve gases) has been developed in ASHT II
  • testing revealed that there are many entries in the REACH part of RATOC I (tables 1 to 3) that describe chemical reaction products not relevant for RAS CHEM.  These entries should be deleted from the RAS CHEM Key Chemicals List
  • The ASHT II substances add-on will be delivered by the ASHT team to be included in the key chemicals list soon. A combination of RATOC I and the ASHT II agents list should be named RATOC II.

References

(1) Desel H (2005-07-07) RAS-CHEM - proposal for prototype. Record of project meeting 2005-07-05 in Luxembourg and technical description of the system. Distributed by email to ASHT project members in preparation of the joint meeting of EAPCCT Board and ASHT partners on 2005-10-02 in Brussels.

(2) Schaper A, Desel H (2007) ASHT Project (Phase I) - Evaluation of RAS CHEM version 1.1.2 build 17/03/2006 (prototype) 2007-02-12. Available at: http://www.giz-nord.de/php/images/stories/asht/ras-chem_eval_2007-02.pdf (last test: 2009-12-11)