Performance Assessment Workbench

Purpose

The Performance Assessment Workbench (PAW) is a knowledge management tool used to integrate assessment and feedback into learning objects within the Knowledge Warehouse (KW). PAW enables instructors, designers, or training managers to plan, design, create, and manage tests and feedback efficiently. PAW provides a mechanism to tailor students’ learning, track their progress, and evaluate the training process.

Implementation Considerations

PAW is designed to support business processes as a(n):

  • Assessment engine, supporting examinations and appraisals within personnel management process
  • Integrated authoring environment for test items and tests
  • Platform for creation, execution, and evaluation of web-based surveys
  • Environment for student-oriented education, improvement, and feedback
  • eCommerce solution for learning technology products, such as training and tests
  • High-end reporting system for all types of feedback-oriented systems

Integration

PAW complements the SAP Knowledge Warehouse to form an integrated online learning environment. It enables users to develop, administer, and use web-based training strategies.

Features

PAW offers users the following features:

  • Author component to create and maintain test content
  • Player component to execute tests online and offline
  • Proctor component to monitor test events
  • Manager component with the following online functions:
    • Test administration
    • Results management
    • Reporting


Test

Definition

A test is generally a series of questions or exercises used to measure the skill, knowledge, level of understanding, or capacities of an individual or group. It is the basis for evaluation.

Use

There are several different methods of testing suited to various situations. The Performance Assessment Workbench (PAW) supports the following methods of testing:

  • Certification Testing
  • Self-testing
  • Survey (though not an actual type of test, survey is included here since it is evaluated similarly to other test methods)

Structure

A test structure may consist of:

  • Sub-tests, usually containing related performance units
  • Performance units, containing various test items
  • Test items

You specify various attributes for a test. These attributes determine how the test is handled. Each of the test structure components have their own attributes.


Test Item

Definition

A test item, also called simply ‘item’, is a specific task defined in a test together with the possible solutions for the task. The Performance Assessment Workbench (PAW) supports various types of item formats depending on the task to be performed.

Use

In general, a multiple choice (MC) item poses a question to the participant in the item stem. The participant is instructed to select the correct answer(s) from a list of at least two answer alternatives. There are three types of MC items available in the Knowledge Workbench:

  • MC1 is a multiple-choice item with one correct answer and several incorrect answers, called distracters. You can also create true/false items using the MC1 item format. In true/false items, the participant must make a decision for one of two answer alternatives.
  • MCN is a multiple-choice item with two or more correct answers.
  • MC explicit is a multiple-choice item in which each answer alternative is either correct or incorrect. Usually with MC items, it is possible for the participant to narrow down the correct answer by ruling out incorrect answers. MC explicit format prevents this from happening. With an MC explicit item, the participant must specify for each answer alternative whether the answer is correct or incorrect. This type of item provides you with additional information about specific knowledge gaps that the participant may have.

A rating item is what you might find in a questionnaire. It is similar to an MC1 item in that the participant may select only one answer alternative. In contrast to the MC1 format, however, there is no correct answer. Each answer alternative is assigned a score. The participant rates or scores the item content according to a multiple-level scale defined by the author. Unlike in a typical test scenario, the answer alternatives in a rating item are seldom weighted equally but rather in ascending or descending order. You can only use rating items in the test category Survey to collect participant opinions and evaluations.

A fill-in item has an open format, allowing the participant to type an answer in a text field instead of choosing from a list of answers. No answer alternatives are displayed.

Structure

An item consists of a question, statement, or similar and two or more answer alternatives.

Integration

A test consists of one or more performance units. You can group similar performance units in a sub-test. You assign test items to performance units in the Knowledge Workbench.

See also:

PAW Author

Creating an Item Info Object


Certification Testing

Prerequisites

You have specified certification as the scenario for the test type. For more information, see Maintaining Details for a Test Type .

Process Flow

  1. Participants register for a certification event as required.
  2. Before the certification event, the proctor retrieves the booking data for the event directly from the SAP system.
  3. The proctor generates logon information for each participant and distributes it accordingly.
  4. Participants start the test player and enter their ID and password to access the test(s).
  5. After reading the test introduction, the participants run the test under supervision of the proctor.
  6. Upon completion of the test, participants may view feedback on the results.
  7. When all tests are completed, the proctor collects the results, adds comments and certain indicators, if necessary, and transfers the results to the PAW Manager where the results are evaluated.
  8. Once the test results are evaluated, qualifications are generated and certificates can be issued to participants.

See also:

Test Administration with PAW

Test Proctoring with PAW


Test mit Ergebnisspeicherung

Einsatzmöglichkeiten

Mit Hilfe eines Online – Test mit Ergebnisspeicherung können Sie zu jedem Zeitpunkt einen Test durchführen. Ergebnisse werden im PAW Manager gespeichert. Es wird der Veranstaltungstyp angelegt, jedoch keine Veranstaltung. Es ist keine Anmeldung erforderlich.

Voraussetzung

Sie haben den Online - Test mit Ergebnisspreicherung als Testzenario in den Testtypdetails angegeben. Stellen Sie am Test, mit dem PAW Author, die entsprechenden Attribute ein. Die Attribute können nachträglich mit SI80 geändert werden. Weitere Informationen finden Sie unter Details für Testtypen pflegen .

Ablauf

  1. Ein Benutzer öffnet die Web – Seite, die Online – Tests anbietet, und wählt einen Test aus.
  2. Der Test wird aus dem Knowledge Warehouse heruntergeladen.
  3. Der Benutzer arbeitet den Test mit dem browser – gestützen Player durch.
  4. Der Benutzer sieht die Testergebnisse und das Feedback nach Testabschluß an.
  5. Der Benutzer druckt die Ergebnisse bei Bedarf aus, bevor er den Test verläßt.

Die Testergebnisse werden beim Verlassen gesichert.


Self-testing

Purpose

You can run a self-test to evaluate your knowledge or performance in a particular subject without recording the results. No event is created and you do not need to register for a self-test.

Prerequisites

You have specified self-test as the test scenario in the test type details. For more information, see Maintaining Details for a Test Type .

Process Flow

  1. A user accesses a web page that offers self-tests and selects a test.
  2. The test is downloaded from the Knowledge Warehouse.
  3. The user executes the test using the browser-based player or a downloaded offline player.
  4. The user views the test results upon completion of the test.
  5. The user can print test results before exiting.

The test results are not saved upon exit.


Survey

Purpose

Although similar in structure to a test, a survey has no correct or incorrect answers. Surveys are used to collect information, opinions, votes, or similar.

Process Flow

  1. The user accesses a web page and selects a survey.
  2. User authorization for the survey may be required depending on existing sales or user master record, or previous Training and Event Management (TEM) booking. If no record exists for the user, it is possible to create a new master record or to use an anonymous number generated for the user.
  3. The survey is downloaded from the Knowledge Warehouse.
  4. The user executes the survey using the browser-based test player or a downloaded offline player.
  5. Upon completion of the survey, the raw data is saved under a unique key.


PAW Author

Purpose

The Performance Assessment Workbench (PAW) Author is a tool used for test design and implementation.

Implementation Considerations

The PAW Author offers training and test developers the ability to:

  • Develop and maintain tests using templates for various test formats
  • Develop and maintain tasks or test items
  • Develop and maintain the content to be tested, called performance units
  • Create and maintain test pools, item pools, and folders
  • Export and import information objects and translations, as well as information and tests from non-SAP sources

Integration

The PAW Author is the test editor started from the Knowledge Workbench, the tool used to create and maintain information objects in the SAP Knowledge Warehouse (KW).The PAW Author stores information objects and administrative data in the KW.

Features

The PAW Author offers the following features:

  • Configurable item and answer scoring methods
  • Wizard-aided creation of performance units

See also:

Test Development with PAW


Test Development with PAW

Purpose

One of the key tools that the Performance Assessment Workbench (PAW) offers is an author component integrated in the Knowledge Workbench. You use this tool to create and maintain tests, test items and performance units.

Process Flow

  1. You define individual test objectives and prerequisites, called performance units. For more information, see Creating a Performance Unit Info Object .
  2. You create the questions or tasks, called test items or simply items, that are associated with a performance unit. For more information, see Creating an Item Info Object .
  3. You create a performance unit container to establish a link between the performance unit and the test items.
  4. You assign the items to the performance unit container.
  5. You create the test and then add performance unit containers to the test. You can group related performance units into sub-tests.
  6. You can create multiple test forms, for example by varying the test items or their structure. For more information, see Creating a New Test Form .
  7. You define attributes for items, performance units, sub-tests, and tests. For more information, see Item Attributes , Sub-test Attributes , and Test Attributes .
  8. You determine when and what type of feedback is to be given to test participants. You decide whether feedback is given for individual answers, test items, sub-tests, performance units, or simply for the whole test. You assign an info object containing the feedback to the test.

See also:

Editing a Test Info Object


Creating a Performance Unit Info Object

Use

A performance unit is the smallest measure of performance in a specific content area. It defines the objective to be reached after successful completion of a learning unit, or a prerequisite for a test. It is assigned as a competency in a specific area of learning.

Tests created with the Performance Assessment Workbench (PAW) consist of one or more performance units.

Prerequisites

You have set the appropriate context and created a folder with area Performance Assessment in the Knowledge Workbench Explorer under your local repository.

Procedure

  1. Select the folder in which you want to create the info object.
  2. Choose Info Object ® New ® Info Object….

    3. The Create New Info Object wizard appears.

  3. Select info class Performance Unit.

    4. Performance unit is automatically assigned as the information category and information class.

  4. Choose Next and then enter the general attributes for the info object, that is, the technical name and title of the info object.
  5. Enter all attribute values for the info object to be created, including the following Performance Assessment Workbench (PAW) specific attributes:
    1. Enter a Label with a maximum of four characters for the performance unit.
    2. Specify the Level 2 status of the info object. The status is an attribute defined by individual customers as an enhancement to level 1 statuses. Level 2 statuses are used in the process control when performance units or items are created.
    3. Specify the Owner of the info object. The owner is a person other than creator and responsible person, for example a person responsible for the budget. The Owner must have a valid user ID in the SAP System.
    4. Specify the Target group of the performance unit. The list of possible entries is configurable for customer-specific requirements.
    5. Enter the performance unit Version.

The attributes you specify here are general info object attributes, not to be confused with the performance unit attributes you specify when editing the performance unit.
  6. After you have entered the detailed attributes, choose Finish.

The wizard closes and the info object is created.

Result

Once you have created the performance unit information object, you can edit its contents.

See also:

Editing a Performance Unit Info Object

Creating a Performance Unit Container


Editing a Performance Unit Info Object

Prerequisites

You have already created the info object in the Knowledge Workbench Explorer.

Procedure

  1. Select the info object in the Knowledge Workbench Explorer and then choose Info Object ® Edit Info Object(s).

    2. The Performance Assessment Workbench (PAW) Author component opens with an empty performance unit template.

  2. Enter the performance description and other information as required, or use the wizard to help you formulate this information.
  3. Choose Attributes to specify the performance unit attributes.

    4. The attributes for a performance unit are similar to those for a test. For more information, see Test Attributes .

  4. Save your entries and then exit the author component.

    5. The info object is saved in your local repository.

  5. Select the info object in the local repository folder and choose Update Info Object….

The info object is checked in to the Knowledge Warehouse.

Result

Once you have created and edited the performance unit, you can create a performance unit container, which links items to the performance unit. For more information, see Creating a Performance Unit Container .


Creating an Item Info Object

Use

Test items are the individual elements that make up a test or performance unit. An item consists of a test question, statement, exercise, or similar with possible answer alternatives. You can create various types of items within the Knowledge Workbench. The Knowledge Workbench currently supports three multiple choice (MC) variants, a rating and fill-in item formats for the Performance Assessment Workbench (PAW). For more information about the various item formats, see Test Item .

Procedure

  1. In the Knowledge Workbench Explorer, choose Info Object ® New ® Info Object… to open the Create New Info Object wizard.
  2. Select Item as the info category.
  3. Select the item format you want to use as the info class and then choose Next.
  4. Enter the attribute values for the item and then choose Finish.

The item you created appears in the Knowledge Workbench Explorer.

The item attributes you specify here are general attributes for the info object, not to be confused with the item-specific attributes you specify when editing the item.

Result

Once you have created the item, you can edit its contents.

See also:

Editing an Item Info Object

Creating a Media Info Object

Assigning an Item to a Performance Unit Container


Editing an Item Info Object

Use

Since each item format requires the participant to perform different activities, the method you use to edit the item info objects differs depending on the format you choose for the item. For more information about the various item formats, see Test Item .

Prerequisites

You have already created the info object in the Knowledge Workbench Explorer and assigned an item format as the info class.

Procedure

  1. Select the info object in the Knowledge Workbench Explorer and then choose Info Object ® Edit Info Object(s).

    2. The Performance Assessment Workbench (PAW) Author component opens with an empty item template of the format you selected when you created the item. Each template contains informative text in the entry fields and tool tips to guide you through the editing procedure.

  2. Choose Attributes to maintain the item attributes. For more information, see Item Attributes .
  3. (Optional) Choose Options -> Enable encrypting to encrypt item texts.

    4. If you do not encrypt the items, anyone with access to the items can read the text in an editor, such as an XML editor or an internet browser.

  4. Enter the question, statement, exercise, or similar.
  5. Enter an instruction for this item directed at the participant. For example, you can specify whether the participant must select one or more than one answer in a multiple-choice item, or warn the participant that the answer he enters is case-sensitive.
  6. Enter the answer(s) to the question or statement depending on the item format:
    • MC1: Mark only one correct answer by clicking on the button before the appropriate answer.
    • Rating: Enter a positive value as the item weight, since there are no incorrect answers in a questionnaire.
    • MCN:
      1. Mark each correct answer by clicking on the button before the appropriate answer.
      2. Specify the score for each answer alternative. Make sure that at least one answer alternative has a score greater than 0.

For example, as is common practice you might specify that the correct answers have a score of 1 and the incorrect answers have a score of 0. In this case, however, it is possible for the test participant to receive the maximum item score simply by choosing all answer alternatives.

Instead you could specify that both correct and incorrect answers have a score of 1. In this case, a positive score is given for selecting the correct answer as well as for not selecting the incorrect answer.
    • MC explicit: Enter weighting values as with MCN items.
    • Fill-in:
      1. Add possible synonyms that the participant might enter as the correct answer. For example, you can specify that the word four or the number 4 can be entered.
      2. In the item editor, choose Show synonym to see synonyms you have entered for the item. Choose Add synonym to enter a new synonym. To delete a synonym, select the synonym from the list and then choose Delete synonym. Choose Undo delete if you mistakenly delete an entry.

If you select Match case, the participant must enter the answer exactly as you have specified in the item, for example using the same uppercase and lowercase letters.
  7. Save your entries.

Result

You can add edited items to a test or a performance unit container.

See also:

Assigning an Item to a Performance Unit Container

Editing a Test Info Object


Creating a Performance Unit Container

Use

A performance unit container is an information object that establishes a link between items and a performance unit. A performance unit container is technically a structure referencing all of the items that represent the performance unit.

Procedure

  1. Choose Info Object ® New ® Structure object… to open the Create New Structure Object wizard.
  2. Select Performance Unit Container as the information category and then choose Next.
  3. Select a root info object as follows:
    1. Choose Create New >>> if you want to create a new performance unit to be used as the root object.

The Create New Info Object wizard appears in which you can create the performance unit.

OR
    1. Choose Select Info Object >>> to select an existing performance unit.

      2. The Locate Info Object dialog box appears.

    2. Choose the method you want to use to search for the object and then choose Continue.
    3. Specify additional search criteria on the dialog box that appears and then start the search.
    4. Choose the performance unit you want from the search results and then choose OK.
  4. Enter the technical name and title as well as all attribute values for the structure object to be created.
  5. After you have entered the detailed attributes, choose Finish.

The wizard closes and the structure info object is created.

Result

Once the structure object is created, you can open the structure editor to insert the necessary info objects.

For more information, see Assigning an Item to a Performance Unit Container .


Assigning an Item to a Performance Unit Container

Use

A performance unit container is a structure object that links a performance unit with test items. To establish this link, you must assign test items to the performance unit container.

Prerequisites

You have already created the performance unit container and any items you want to assign to it.

Procedure

  1. In the Knowledge Workbench Explorer, select the performance unit container.
  2. Choose Info Object ® Edit Info Object(s).

    3. The structure editor appears.

  3. Drag and drop the desired test items into the structure editor for the performance unit container.

Result

A link between the performance unit and test items is established. You can add performance unit containers to tests. For more information, see Editing a Test Info Object .


Creating a Test Info Object

Procedure

  1. Choose Info Object ® New ® Info Object… to open the Create New Info Object wizard.
  2. Select info class Test.

    3. Test is automatically assigned as the information category and information class.

  3. Choose Next and then enter the technical name and title of the info object.
  4. Enter all general attribute values for the info object to be created, including the following Performance Assessment Workbench (PAW) specific attributes for test info objects:
    1. Specify whether Booking is necessary for the test, that is whether a test participant must register for the test.
    2. Specify whether the test is to be Proctored or not.

The attributes described here are general info object attributes, not to be confused with the test-specific attributes you define in the actual test info object. For more information about these attributes, see Test Attributes .
  5. After you have entered the general attributes, choose Finish.

The wizard closes and the info object is created.

Result

Once you have created the test information object, you can edit its contents. For more information, see Editing a Test Info Object .


Editing a Test Info Object

Use

When you create a test, you must specify how the test is structured. The test structure may consist of the following elements:

  • Test
  • Sub-test
  • Performance unit

You use the Performance Assessment Workbench (PAW) Author component to edit the test from the Knowledge Workbench.

Prerequisites

You must have already created the test info object in the Knowledge Workbench Explorer.

Procedure

  1. Select the test info object in the Knowledge Workbench Explorer and then choose Info Object ® Edit Info Object(s).

    2. The PAW Author component opens with an empty test template.

  2. If necessary, insert sub-tests by choosing Test ® New Sub-test.

    3. A sub-test icon appears under the test in the structure.

  3. Insert performance units in the test structure. You can organize performance units and items under tests or sub-tests.
  4. Drag and drop a performance unit container to a sub-test to assign the items within the container.

    5. All items in the performance unit appear in a selection list.

  5. Choose the items you want to assign to the test structure.
  6. Select the test object and then choose Test ® Attributes….

    7. The Edit test attributes dialog box appears.

  7. Specify the attributes for the test. For more information, see Test Attributes .
  8. Insert an introduction info object for the test by dragging the info object and dropping it on the field Introduction title in the Edit test attributes dialog box.

You create the introduction info object similarly to other info objects but with info class Generic. You can use the introduction template to create your own introductions.
  9. Insert a feedback info object for the test by dragging the feedback info object and dropping it on the field Feedback template title in the Edit test attributes dialog box.

You create the feedback info object similarly to other info objects but with info class KWPWFTE. For more information, see Test Feedback Creation . There are two feedback templates ( Feedback Template A and Feedback Template B ) that you can use to create feedback info objects.
  10. Specify the attributes for the sub-test. For more information, see Sub-test Attributes .
  11. Display attributes for performance unit and item, if necessary. You can edit the attributes for performance units and items directly in the respective object.

Result

You can create different versions of the same test. For more information, see Creating a New Test Form .


Test Feedback Creation

Purpose

You create a feedback template to determine how test feedback should appear to the test participant upon completion of the test. You can create your own feedback templates or you can use one of two standard feedback templates ( Feedback Template A and Feedback Template B ) in the Performance Assessment Workbench (PAW).

The standard templates are divided into several areas. Each of these areas contains certain placeholders which are replaced by values in the completed test after the initial test evaluation in the PAW Player. You can delete the placeholders in the template if you do not want the feedback to include a particular value. For more information about the placeholder values, see Test Feedback Placeholders .

Process Flow

  1. You use one of the standard templates to create feedback for a test or you create your own feedback template. You create the feedback as a Microsoft Word document.
  2. You create the feedback info object in the Knowledge Warehouse Explorer and assign info class KWPWFTE to it.
  3. You assign the feedback info object to a test as an attribute of the test. The content of the feedback is ultimately determined by the settings in the test attributes. For more information, see Test Attributes .

Result

When the test participant completes the test to which you assigned the feedback info object, the participant can view the feedback you specified.


Creating a New Test Form

Use

You create test forms to develop different versions of the same test.

Prerequisites

You have already created the test info object. The test structure editor is open.

Procedure

  1. Select the test icon in the test structure from the Configure Test Structure frame on the left half of the window.
  2. Drag and drop the test structure into the Create Test frame on the right half of the window.
  3. To create an additional test form or a parallel test, choose Test ® New test form.
  4. Repeat the drag and drop procedure for the new test form.
  5. Delete items that should not be included in the test from the structure for each test form.
  6. Save your changes.
  7. Check the test info object into the Knowledge Warehouse from the local repository.


Creating a Media Info Object

Use

You can include a graphic, screenshot, video, or similar media object in an item. To do this, you must create a media info object.

Prerequisites

The media file you want to insert must be in .DOC format. If the file you want to include is in another format, save the file in a Microsoft Word document.

Procedure

  1. Choose Info Object ® New ® Info Object….

    2. The Create New Info Object wizard appears.

  2. Select Media as the info class.

    3. Media is automatically assigned as the information category and information class.

  3. Choose Next.
  4. Enter the attributes for the info object.
  5. Drag the media document from a second Knowledge Workbench Explorer window and drop it on the Media description attribute field.
  6. Choose Finish.

The media info object is created.


PAW Manager

Purpose

The Performance Assessment Workbench (PAW) Manager is a tool for the administration of information associated with tests. The name PAW Manager or simply manager refers to the PAW functions that take place in the SAP System.

Implementation Considerations

You use the PAW Manager to:

  • Create, customize, and maintain data for events, tests, and participants
  • Execute reports and analyze test results in many formats using various criteria
  • Manage test results in cooperation with a connected HR Personnel Development system

Integration

The PAW Manager accesses the Human Resources (HR) system for personnel and Training and Event Management (TEM) data. Test result information is transferred to HR and qualifications are assigned.

In addition, you can also transfer data for use in external programs, such as Statistical Product and Service Solutions (SPSS) or other statistical evaluation software.

Features

  • Data consistency between systems
  • Ability to monitor individual users’ progress
  • Flexible reporting at event, test, or participant level according to customer requirements
  • Quality assurance through comprehensive statistics

See also:

Test Administration with PAW

Displaying Test Results

Assigning Qualifications for Test Participants

Executing Statistical Evaluation Reports

Defining PAW Authorizations


Test Administration with PAW

Purpose

As a test administrator, you must specify which tests are given at an event. You must also establish a connection between an event, tests, and a location in the Performance Assessment Workbench (PAW).

Prerequisites

Event and location data must already exist in the HR Training and Event Management (TEM) system. Test information object have to exist in the Knowledge Warehouse. Participants of a test-event also need to exist in the TEM system in order to be booked on the event. Furthermore, if the system holding qualification information is different from the TEM system, the participant must also be existent in that particular system, otherwise no qualification can be assigned.

Process Flow

  1. You specify the name of the test, called the test type. For more information, see Assigning Test Types from TEM .
  2. You enter test type details , such as qualifications to be assigned after successful completion of test. You also assign a test information object to the test type. This information object contains the actual test contents.
  3. You create a test catalog for a location. For more information, see Defining a Test Catalog .
  4. You enter test catalog details, such as test types that are given at the location.

Result

The information you entered for the test type is accessed anytime the test is given. This information and the test catalog details are used in the PAW Proctor when a certification event or similar supervised test event takes place. For more information, see PAW Proctor .

After tests are completed, you can display test results and assign qualifications to participants. You can also run reports to evaluate statistics from the test results. If necessary, you can export the statistics to external programs for further evaluation.

See also:

Displaying Test Results

Assigning Qualifications for Test Participants

Executing Statistical Evaluation Reports

Maintaining PAW Settings


Assigning Test Types from TEM

Use

You must specify which tests are given at an event. To do this, you must first define test types by assigning the test types from the HR Training and Event Management (TEM) system and then enter details for them.

Prerequisites

The tests you want to assign must already exist in the TEM system.

Procedure

  1. From the Performance Assessment Workbench (PAW) initial screen, choose Administration ® Define test type.

    2. The PAW Test Type Definitions screen appears.

  2. Choose Add test types to access test types in the TEM system.
  3. In the dialog box that appears, enter selection criteria, if necessary, and then choose Continue to start the search.

    4. A list of selection results appears in a new dialog box.

  4. Select one or more test types and then choose Copy.

The test types you selected appear in the list on the PAW Test Type Definitions screen.

Result

You can enter test type details for test types that appear in the list. For more information, see Maintaining Details for a Test Type .


Maintaining Details for a Test Type

Use

You must specify which tests are given at an event. To do this, you must first define test types by assigning the test types from the HR Training and Event Management (TEM) system and then enter details for them.

The test type details contain important information about tests. For example, a test proctor requires this information to prepare, monitor, and complete a test event successfully.

Prerequisites

You have assigned the test type from the TEM system in the Performance Assessment Workbench (PAW). For more information, see Assigning Test Types from TEM .

Procedure

  1. From the PAW Test Type Definitions screen, select a test type from the list and then choose Details.
  2. On the Settings tab, choose whether the test type is certification test, or self-test .
  3. On the Test info object tab, create a link between the test type and a test information object. For more information, see Assigning a Test Info Object to a Test Type .
  4. On the Results handling tab, specify which results management functions can be carried out under which circumstances.

    5. For example, if test results should only be transferred when no problems occurred, you mark only the green checkbox under Transfer.

  5. On the Thresholds tab, specify percentages for which successful completion of the test is rated.
  6. Assign a proficiency rating to the qualification that is linked to the test type.
  7. Save your entries.

See also:

Defining a Test Catalog


Assigning a Test Info Object to a Test Type

Use

The test info object contains the test content to be used at an event. You must assign at least one test info object to a test type. You do this when maintaining the test type details.

Prerequisites

The test info object you want to assign already exists in the Knowledge Warehouse. For more information about how to create a test info object, see Creating a Test Info Object .

Procedure

  1. On the PAW Test Type Details screen, choose the Test info object tab and then choose Add info object.

    2. The Info Category dialog box appears.

  2. Choose the information category of the test object you want to assign to the test type and then choose Copy.
  3. In the dialog box that appears, enter selection criteria and then choose Execute to search for the test object in the Knowledge Warehouse.

    4. A list of selection results appears.

  4. Select one or more info objects to which you want to link and then choose Copy.

    5. The knowledge link is created.

  5. If necessary, start a new search to find information objects that meet different criteria.
  6. Enter the dates for when the information object link is valid.


Defining a Test Catalog

Use

A test catalog contains information about test locations and the tests to be conducted at the test locations. It can be used, for example, by a test proctor who requires this information to prepare, monitor, and complete a test event successfully.

Prerequisites

The test locations and test types you want to add to the test catalog already exist in the HR Training and Event Management (TEM) system.

Procedure

  1. From the Performance Assessment Workbench (PAW) initial screen, choose Administration ® Define test catalog.

    2. The PAW Test Catalog Definitions screen appears.

  2. Choose Add locations to access locations in the TEM system.
  3. In the dialog box that appears, enter selection criteria, if necessary, and then choose Continue to start the search.

    4. A list of selection results appears in a new dialog box.

  4. Select one or more locations and then choose Copy.

    5. The locations you selected appear in the list on the PAW Test Catalog Definitions screen.

  5. Select a location from the list and then choose Details.
  6. On the Test types tab, create a link between the location and a test type:
    1. Choose Add test type.
    2. On the dialog box that appears, enter selection criteria and then choose Enter.

      3. A list of selection results appears.

    3. Select one or more test types to which you want to link and then choose Copy.
    4. If necessary, start a new search to find information objects that meet different criteria.
    5. Enter the dates for when the information object link is valid.
  7. Save your entries.

Result

You can display all test catalogs from the Performance Assessment Workbench main screen by choosing Administration ® Display test catalog.


Displaying Test Results

Use

You can display an overview of all test results based on selection criteria you specify or you can display the results for individual test participants.

Procedure

You access the individual functions shown in the table from the Performance Assessment Workbench (PAW) main menu.

Function | Follow-on menu path | What you should know
Display results overview | Results management ® Display overview | Enter selection criteria to limit results to be displayed and choose Execute. You can manipulate the overview to display data in various formats by using functions under the Display menu. For example, you can restructure the display tree or sort participants by various criteria.
Display results for participant | Results management ® Display participant | Enter participant identification and choose Execute. You can view individual test results for participant, qualifications earned, and other information.

See also:

Assigning Qualifications for Test Participants

Executing Statistical Evaluation Reports


Assigning Qualifications for Test Participants

Use

Qualifications are created and stored in the HR Personnel Development (PD) system, where they can also be assigned to persons. You can assign qualifications to test types in the Performance Assessment Workbench (PAW).

When a test participant successfully completes the test to which the qualification is assigned, qualifications can be generated in the corresponding HR PD system, depending on the settings of the test-types involved. The system runs the qualification assingment automatically after the import of the results into the PAW database. However, there is an additional report that you can use to manually assign the qualification to the participant and update the HR system with the newly acquired qualifications.

Prerequisites

  • The participant has met all minimum requirements to receive the qualification as specified in the test type and in the PD system.
  • The settings in the test type allow the qualifications to be transferred.

Procedure

The individual functions in the table are accessed from the Performance Assessment Workbench (PAW) main menu.

Function

Follow-on menu path

What you should know

View results and transfer qualifications

Results management ® Assign qualification

  1. Enter the selection criteria for which you want to search events.
  2. Specify the program mode. Here you determine whether events found in the search are displayed first before the qualifications are generated and transferred (simulation mode) or not. From the display, you can trigger the qualification generation and transfer results to HR. If you do not choose simulation mode, the qualifications are automatically generated and transferred to HR.
  3. Specify which flags are to be taken into consideration during transfer. For example, if you select "Pass" flag, the system checks to make sure the participant passed the test before transferring the qualifications.

Generate qualifications in batch report

Results management ® Generate qual. batch

Enter the selection criteria for which you want to search events and specify exceptions.

Result

You can display a log for each report you run:

Function | Follow-on menu path | What you should know
View transfer log by calendar month | Results management ® Display transfer log | Enter date range and location ID.
View batch report log | Results management ® Display batch prot. | Enter event ID.

See also:

Executing Statistical Evaluation Reports


Executing Statistical Evaluation Reports

Use

You can execute statistical reports to evaluate the results from individual tests or items. You can use these reports, for example to determine how effective a test is. In addition, you can also export test data to be evaluated with third-party software.

Procedure

The individual functions in the table are accessed from the Performance Assessment Workbench (PAW) main menu.

Function | Follow-on menu path | What you should know
Execute standard report | Statistical eval. ® Display descr. stat. | Enter the test information objects for which you want to display statistics. Enter other selection criteria as needed to limit the search results. The test result statistics that are listed are standard statistical criteria, such as range, median, mean, and standard deviation.
Export test results | Statistical eval. ® Display answ. detail | Enter the test information objects whose results you want to export and any additional selection criteria.
Execute report for test items | Statistical eval. ® Display item results | Enter the test information objects for whose items you want to display statistics. Enter other selection criteria as needed to limit the search results.
Execute report by test structure | Statistical eval. ® Display test structure | Enter test name.


Maintaining PAW Settings

Use

You can define, change, and delete Performance Assessment Workbench (PAW) operational settings to determine how PAW and its data are handled. You must first define the setting and then maintain values for the setting.

Procedure

  1. From the PAW main menu, choose Settings ® Maintain values.

    2. The PAW Maintain Settings screen appears.

  2. Choose Create to define a new setting definition.
  3. Enter the required information for the new setting:
    1. Specify the system client, setting name, and description.
    2. In the Value type field, enter S if the setting has only one single value or M if the setting can have multiple values.
    3. In the Origin field, specify whether the setting is system standard setting or a customer-specific setting.
    4. Specify whether the setting can be modified or not.
  4. Choose Save definition.

    5. The new setting appears in the structure on the left side of the PAW Maintain Settings screen under the origin you specified.

  5. Double-click the definition in the structure.

    6. The setting information appears in the top right portion of the screen.

  6. In the bottom right portion of the screen, choose the Create icon to define a new value for the setting you defined.

If the setting definition specifies that only one value can exist, you cannot create more than one value for the setting.
  7. Enter the required information for the new value and then choose Save value.
  8. To change a setting definition, select the setting in the structure and then choose Change.

    9. The setting information appears in the top right portion of the screen where you can make changes.

  9. To delete a setting definition, select the setting in the structure and then choose Delete definition.

The definition and its values are deleted. The definition no longer appears in the structure.

Result

You can view all settings maintained for PAW by choosing Settings ® Display values from the PAW main menu.

See also:

Defining PAW Authorizations


Defining PAW Authorizations

Use

In addition to authorizations granted through the roles concept, the PAW Manager allows to restrict acces to test-result data on a data-level, which is, access can be restricted according to certain profiles that combine a number of either

locations

test-types

test info-objects

For standard reports location- and test-type profiles are used. For statistical evaluation this is also the case. However the system can be set up (using settings functionality) authorize access to test info-objects instead of test-types for statistical evaluations only.

Data-level authorization must be turned on using the settings functionality (Maintaining PAW Settings [Page * ])

Procedure

You first define data-access profiles that contain the identifiers of locations, test-types or test-info-objects. Then you assign these profiles to a user (the user profile).

You can access the following functions from the Performance Assessment Workbench (PAW) main menu.

Follow-on menu path | What you should know
Authorizations ® Define for location | Enter profile and location ID.
Authorizations ® Define for test type | Enter profile and test type ID.
Authorizations ® Define for test IO | Enter profile and test ID.
Authorizations ® Define for user | Enter user ID and profile and then specify authorization type.
Authorizations ® Display auth. report | Enter user ID.

Result

If data-level access authorization is switched on, access to test results will be restricted to those that have the appropriate data-access profiles for the desired data.


PAW Player

Purpose

The Performance Assessment Workbench (PAW) Player is a tool used by the participant to run the test. Similar in appearance to the PAW Author, the PAW Player provides an easy-to-use medium for test participants to read and answer test items, and to monitor and evaluate test progress and results.

Features

  • Ability to run online through SAP systems and intranet/internet
  • Ability to run offline without a connection to SAP systems
  • Restart function that enables the test participant to continue after system disturbance or failure
  • Ability to use and display multimedia files, such as screencams, and audio files

Since the design and use of the PAW Player is easily understandable, no extended documentation exists for the tool. When the PAW Player is started, an introductory document for the individual test is automatically displayed. This introduction provides the test participant with information specific to the test as well as instructions on how to use the player.

See also:

Introduction Template


PAW Proctor

Purpose

The Performance Assessment Workbench (PAW) Proctor component is a tool used to manage and monitor certification events or similar proctored test environments. Generally, a proctored test is when several test participants attend a supervised event to complete a test on a computer.

Integration

The PAW Proctor communicates with the HR Training and Event Management (TEM) system through the PAW Manager to download registration and event information, and to upload test results. The PAW Proctor also communicates with the Knowledge Warehouse (KW) to ensure that the required information objects for a certification event are available, and if necessary, to download them.

Features

The PAW Proctor provides the following:

  • Equal conditions for all test participants
  • Verification of participant registration anywhere within the connected network
  • Ability to download current registration and event information from the Training and Event Management (TEM) system
  • Last-minute participant registration
  • System checks, ensuring that all required information objects are downloaded before certification event
  • Control access to tests
  • Restart capability in case of system failure
  • Control of availability, storage, and transfer of test results

See also:

Test Proctoring with PAW


Test Proctoring with PAW

Purpose

You can use the proctor component of the Performance Assessment Workbench (PAW) to administer tests in a controlled and supervised test environment. As the proctor of such an event, you must ensure that the test conditions for all participants are equal.

Process Flow

  1. Before the certification event, you retrieve the booking data for the event from the PAW Manager. The PAW Manager accesses the event and participant data directly from TEM. The data includes the event date, location, and participants, as well as the tests to be taken by each participant. For more information, see Uploading Data to the PAW Proctor .
  2. The list of tests required for the event is updated to accommodate exceptions. For example, the list is changed accordingly if an FI test for Europe is requested at a certification event in Boston, where the FI test for the US is the default.
  3. You generate logon information for each test participant. During the certification event, participants must enter their identification and password to access the test(s).
  4. You start and stop the events and monitor the tests progress during the certification event. If necessary, you can pause and then resume the tests once they have been started.
  5. When all tests are completed, you collect the results, add comments and set signals, if necessary.
  6. You then transfer the test results to the PAW Manager where the final results are evaluated.

See also:

Starting the PAW Proctor

Monitoring Events in the PAW Proctor

Maintaining Participant Information in the PAW Proctor

Monitoring Tests in the PAW Proctor


Starting the PAW Proctor

Procedure

  1. Open the Performance Assessment Workbench (PAW) Proctor.
  2. Choose Proctor as the user.

    3. If you log on to the proctor as a Super user, in addition to all standard functions of the proctor, you can also save test results and delete event data.

  3. Choose the server from which you want to run the proctor.
  4. Enter the password and then choose OK. The PAW Proctor is started.


Uploading Data to the PAW Proctor

Use

The Performance Assessment Workbench (PAW) Proctor offers additional functions for importing and updating test and event data.

Procedure

The individual functions in the table are found under the Data menu in the PAW Proctor.

Function | Follow-on menu path | What you should know
Import new event information from local server | Import ® Event data | Select ZIP file containing all the relevant files for the event.
Update event information from Communication Server System | Update ® Event data… | Reload event data from the Communication Server. This can be a more extensive update which may require additional time.
Update booking information from SAP System | Update ® Booking data… | Reload booking data from HR System. This is a more extensive update which requires additional time.
Download booking data | Download ®Booking data | Reload booking data from HR System. This is a more extensive update which requires additional time


Monitoring Events in the PAW Proctor

Use

You can use the event details provided in the Performance Assessment Workbench (PAW) Proctor to perform various tasks before, during, and after a test event.

Procedure

  1. In the PAW Proctor, choose the Event detail tab and then select the event you want to monitor or change from the list of events in the left frame of the PAW Proctor.

    2. The information for the event that you choose appears in the fields on the Event detail subscreen.

  2. Print test letters with brief event introduction and specific logon information for each event participant.
  3. Use the icons that resemble recorder controls at the top of the subscreen to start and stop the event.

    4. Once you start an event, the participants can log on. When you stop the event, you can no longer edit the event. You can restart the event once it has been stopped using the green flag icon. The Pause event and Resume event functions become active after you have started the event. Once you pause an event, no further participants can log on to this event. After the event is resumed the participants can log on again. The Status field shows whether the event is running, paused, or stopped.

  4. Edit comments for event and set event signal.

    5. The event signal can have varying significance depending on how your company uses it. Generally, red means severe errors, yellow means minor errors, and green means no or insignificant errors.

  5. Save test results (super user only). For more information, see Starting the PAW Proctor .
  6. Delete event, if necessary (super user only after saving test results).
  7. Export event data to diskette, for example, if you have no network connection.

See also:

Maintaining Participant Information in the PAW Proctor

Monitoring Tests in the PAW Proctor


Maintaining Participant Information in the PAW Proctor

Use

You can use the participant details provided in the Performance Assessment Workbench (PAW) Proctor to create, change, or delete individual participant information before, during, and after a test event.

Procedure

  1. In the PAW Proctor, choose the Participant detail tab and then select the participant you want to change or display from the list of participants in the top frame of the screen.

    2. The information for the participant that you choose appears in the fields on the Participant detail subscreen.

  2. If you want to add a new participant to the list, choose Add participant and then enter the required data for the new participant.
  3. Choose Edit to change the participant details.

    4. The fields are ready for input.

  4. Regenerate password, if necessary, and then Show password for selected participant.
  5. If necessary, print the test letter with a brief event introduction and specific logon information for the selected participant.
  6. Assign and remove tests for selected participant.
  7. Save the changes.

See also:

Monitoring Events in the PAW Proctor

Monitoring Tests in the PAW Proctor


Monitoring Tests in the PAW Proctor

Use

You can use the test monitor details provided in the Performance Assessment Workbench (PAW) Proctor to monitor and manage the progress of individual tests before, during, and after a test event.

Procedure

  1. In the PAW Proctor, choose the Test monitor tab and then select the test you want to restart or stop from the list of tests in the top frame of the PAW Proctor.
  2. Choose Stop to end the selected test. Restart test that you have stopped, if necessary.
  3. Import a test from a file if required.
  4. Choose Edit comments to enter comments about the test and to change the test signal.

The test signal can have varying significance depending on how your company uses it. Generally, red means severe errors, yellow means minor errors, and green means no or insignificant errors.

See also:

Monitoring Events in the PAW Proctor

Maintaining Participant Information in the PAW Proctor


Test Feedback Placeholders

The two standard feedback templates ( Feedback Template A and Feedback Template B ) are structured as tables. However, you can structure a template as simple paragraphs if you find the paragraph form easier to work with.

When you try to copy or delete parts of a table, any formatting specified for that area of the table is copied or deleted.

Area Test Placeholders

The test feedback area is enclosed between the markers $TEST_ and _TEST$. This area is not repeated since only one set of test feedback can exist for a test.

Placeholder | Replaced by value | Description
$TEST_DESCRIPTION | Description | Test title
$TEST_SCORE | IScore | Points earned
$TEST_MAXSCORE | IMaxScore | Maximum possible points
$TEST_PERCENTSCORE | IPercentScore | Percent earned
$TEST_URL | FeedbackLinkURL | Link to a URL, for example to training materials for the test. You can also insert the URL to an info object from the Knowledge Warehouse here.
$TEST_URL_DESCRIPTION | FeedbackLinkIODescription | Title of the link specified as the FeedbackLinkURL.
$TEST_SCORINGTABLE_FEEDBACK | ScoringtableFeedback | Text from the scoring table assigned to the percent earned.

Markers

Start | End | Description
$TEST_USE_SCORING_TABLE | TEST_USE_SCORING_TABLE$ | Contents between these markers are only displayed if the test attributes specify that the scoring table is to be used.
$TEST_USE_NOT_SCORING_TABLE | TEST_USE_NOT_SCORING_TABLE$ | Contents between these markers are only displayed if the test attributes specify that the scoring table is not to be used.
$TEST_PASSED_OK | TEST_PASSED_OK$ | Contents between these markers are only displayed if the participant passed the test.
$TEST_PASSED_BAD | TEST_PASSED_BAD$ | Contents between these markers are only displayed if the participant did not pass the test.
$TEST_POSTEDITING_REQUIRED | TEST_POSTEDITING_REQUIRED$ | Contents between these markers are only displayed if the test must be post edited in the PAW Manager, for example if the test includes an essay which cannot be automatically evaluated.

Area Sub-test Placeholders

The sub-test feedback area is enclosed between the markers $SUBTEST_ and _SUBTEST$. This area is reproduced for each sub-test in the test regardless of whether the sub-tests are visible in the test form.

You can hide and show the sub-test feedback area using the Feedback Level attribute in the sub-test. The sub-test area can appear within the test area or outside of it.

Placeholder | Replaced by value | Description
$SUBTEST_DESCRIPTION | Description | Sub-test title
$SUBTEST_SCORE | IScore | Points earned
$SUBTEST_MAXSCORE | IMaxScore | Maximum possible points
$SUBTEST_PERCENTSCORE | IPercentScore | Percent earned
$SUBTEST_URL | FeedbackLinkURL | Link to a URL, for example to training materials for the sub-test. You can also insert the URL to an info object from the Knowledge Warehouse here.
$SUBTEST_URL_DESCRIPTION | FeedbackLinkIODescription | Title of the link specified as the FeedbackLinkURL.
$SUBTEST_CUTOFFVALUE | CutOffValue | Points in percent required to pass the sub-test.
$SUBTEST_SCORINGTABLE_FEEDBACK | ScoringtableFeedback | Text from the scoring table assigned to the percent earned.

Markers

Start | End | Description
$SUBTEST_USE_SCORING_TABLE | SUBTEST_USE_SCORING_TABLE$ | Contents between these markers are only displayed if the sub-test attributes specify that the scoring table is to be used.
$SUBTEST_USE_NOT_SCORING_TABLE | SUBTEST_USE_NOT_SCORING_TABLE$ | Contents between these markers are only displayed if the sub-test attributes specify that the scoring table is not to be used.
$SUBTEST_PASSED_OK | SUBTEST_PASSED_OK$ | Contents between these markers are only displayed if the participant passed the sub-test.
$SUBTEST_PASSED_BAD | SUBTEST_PASSED_BAD$ | Contents between these markers are only displayed if the participant did not pass the sub-test.
$SUBTEST_CUTOFFTEST | SUBTEST_CUTOFFTEST$ | Contents between these markers are only displayed if passing this sub-test is required to pass the whole test.

Area Performance unit Placeholders

The performance unit feedback area is enclosed between the markers $PU_ and _PU$. This area is reproduced for each performance unit in the test regardless of whether the performance units are visible in the test form.

You can hide and show the performance unit feedback area using the Feedback Level attribute in the performance unit. The performance unit area can appear within the test and sub-test areas or outside of them.

Placeholder | Replaced by value | Description
$PU_DESCRIPTION | Description | Performance unit title
$PU_SCORE | IScore | Points earned
$PU_MAXSCORE | IMaxScore | Maximum possible points
$PU_PERCENTSCORE | IPercentScore | Percent earned
$PU_URL | FeedbackLinkURL | Link to a URL, for example to training materials for the performance unit. You can also insert the URL to an info object from the Knowledge Warehouse here.
$PU_URL_DESCRIPTION | FeedbackLinkIODescription | Title of the link specified as the FeedbackLinkURL.
$PU_SCORINGTABLE_FEEDBACK | ScoringtableFeedback | Text from the scoring table assigned to the percent earned.

Markers

Start | End | Description
$PU_USE_SCORING_TABLE | PU_USE_SCORING_TABLE$ | Contents between these markers are only displayed if the performance unit attributes specify that the scoring table is to be used.
$PU_USE_NOT_SCORING_TABLE | PU_USE_NOT_SCORING_TABLE$ | Contents between these markers are only displayed if the performance unit attributes specify that the scoring table is not to be used.


Item Attributes

In addition to the general item attributes automatically assigned to an item through its info category, you can also define detailed characteristics and properties for specific items. You access item attributes by choosing Attributes in the item editor.

The following describes these attributes in detail:

Attribute | Specifies…
Item category | Item category selected when creating the item information object in the Workbench
Beta item | Whether the item is to be used as a beta item, for example during a test phase You can evaluate items with this attribute separately at a later point in the test. For more information about how beta items are handled, see the information in Test Attributes .
Difficulty | Statistical value to be considered when items are evaluated according to a specific frequency of use Enter the value in the text line for this attribute.
Validity | Statistical value to be considered when items are evaluated according to a specific frequency of use Enter the value in the text line for this attribute.
Item total correlation | Statistical value to be considered when items are evaluated according to a specific frequency of use Enter the value in the text line for this attribute.
Item weight | Positive whole number score of the item to be used when evaluating the complete test The default item weight is 1.
Item feedback | Feedback text the test participant can view for this item When you click on the text field for the Item feedback attribute, a dialog box appears in which you can create and edit the feedback text.
Media description | Description of media image, that shows up with item Drag and drop a media object into this field to create a link between item and media


Test Attributes

The test attributes specify the properties or characteristics the test assumes during its execution. The following describes these properties, their possible values, and how tests with these properties are affected.

Attributes

Specifies…

Title

Title entered when creating the test information object in the Workbench

Test category

Test category selected when creating the test information object in the Workbench

Introduction title

Information object with category Introduction created to be used for test instructions as well as for general operational instructions for the test player. You add the instructions to the test by dragging the introduction info object and dropping it on the Introduction title field in the Knowledge Workbench. For an example of an introduction info object, see Introduction Template .

Use test form

Test forms to be used for test execution

Organizational criteria

Elements of the configured test structure, that is sub-tests and performance units, to be used in a test.

If you decide not to use a structure, the test simply consists of a number of items that can be placed in any position within the test. If the test contains structure elements that are sub-tests only and not performance units, the sub-test items can be placed in any position under their corresponding sub-test.

If the test contains both sub-tests and performance units as structure criteria, the performance units can be placed in any position under their corresponding sub-test. In this case, the performance unit items can be placed in any position under their corresponding performance unit.

The same applies if the test contains only performance units and no sub-tests.

If the structure elements of the configured test structure are not saved as structure criteria in the test, the test still retains all characteristics of the test elements it contains. For example, if you created a test with sub-tests that are not specified as structure criteria, the test is not structured by sub-tests in the player, although the test retains the characteristics specified for the sub-tests, such as evaluation and feedback at sub-test level.

Display test structure

Whether the test structure is displayed in the player during test execution

Save test results

Whether individual test results are saved

Maximum test time (min.)

Maximum time available to complete the test in minutes. The test time is displayed as remaining time in the player. The test start, possibly after completion of an introduction, is also the start of the test time. When the test time has run out, the test is automatically ended and no further entries are accepted.

A value of zero for this attribute specifies that the test has no time limit.

Dynamic item selection

Whether test items can be selected dynamically from the test form (attribute: Yes) or only in the sequence defined in the test form (attribute: No). Dynamic selection of test items occurs in the player during test execution.

The attribute values Test, Sub-test, or Performance unit determine the item groups from which items are randomly selected for the test form. The number of possible item groups depends on how the test form is structured.

Item count

Number of pooled items at test level that are to be randomly selected from the test form

Random order

Random ordering of sub-tests, performance units, items, and answer alternatives at test execution.

The attribute value Answer alternatives specifies that the answers for each item in the test should be randomly ordered. The attribute values Item, Performance unit, and Sub-test specify that the respective structure element should be randomly ordered within the next highest node of the test form structure. For example, if a test form has sub-tests and no performance units and you select attribute value Item, the items within a sub-test are displayed in random order.

Participant remarks

Whether participants can add remarks for each item after the test is completed

If remarks are allowed, the participant accesses a text editor by choosing Remarks from the player. The remarks are saved with the participant test results. Participant remarks can provide important information about how effective tests and test items are especially during beta testing phase.

Scoring model

Model to be used by which the individual test results are scored

The value Standard specifies that each correct test item receives exactly one point in score, which means that the maximum attainable test score is equal to the sum of all items in the test. You can adjust the score weight for individual items with the Item weight attribute. SAP recommends the standard setting for practical and test theoretical reasons.

If you select Answer weight, the weights you set in the item editor for individual answer alternatives are used. Scoring models and how they effect test scoring are documented in the item format templates that allow answer weighting.

Scoring process

Whether individual test values of all items in the test are scored using only beta items or using only released items with no beta items. You can test beta items by distributing them in a released operational test. This attribute allows you to gather empirical test data for items that are not yet released.

Cut-off value in %

Whole-number test value that must be achieved to pass the test, as a percentage

If no passing criterion is required, enter a value of zero.

Feedback level

Feedback level of detail

The participant receives feedback for the test once completed. Depending on the test structure, you can specify that feedback for participants is provided at test, sub-test, performance unit, item, and individual answer levels.

Templates are available for feedback at test, sub-test, and performance unit levels. For more information, see the attribute Feedback template title. These templates are specified for each test through the attributes Feedback test passed, Feedback test failed, Feedback web link, and Feedback info object link. If you select these levels, the feedback appears as an HTML page on the participant’s screen once the test is complete.

You enter feedback text for items as an attribute in the item editor. If you want the participant to compare his answers with the correct answers in addition to or instead of seeing the feedback text, you must select the attribute value Answer.

The participant can view the item feedback as the test is displayed again with feedback information.

Feedback test passed

Text in the Feedback template title used when successful completion feedback is provided at the specified level

Feedback test failed

Text in the Feedback template title used when unsuccessful completion feedback is provided at the specified level

Feedback web link (URL)

Web link or URL used in the Feedback template title

The web link enables the participant to access a web page directly from the test, sub-test, or performance unit feedback, for example to find more information about a particular subject where he lacks knowledge.

Feedback info object link

Link to an info object in the Knowledge Warehouse used in the Feedback template title

The info object link enables the participant to access information from the training, documentation, or administration areas of the Knowledge Warehouse directly from the test, sub-test, or performance unit.

Feedback template title

Title of test-specific templates for feedback at test, sub-test, and performance unit level ( Feedback Template A and Feedback Template B )

You add the feedback template to the test by dragging the feedback template info object and dropping it on the Feedback template title field in the Knowledge Workbench. For more information about creating feedback, see Test Feedback Creation .

Use feedback intervals

Whether feedback intervals defined in the scoring table are to be used for the test

If this is the case, the feedback text for the interval is displayed together with the participant test score.

Scoring table

Feedback intervals to provide various feedback texts for defined scoring areas

Proceed as follows:

  1. Click on +++ to open the scoring table.
  2. Click in the window where the table appears.
  3. Edit the lower limit of the first interval and then click on the arrow to confirm the changes.
  4. Edit the lower limit of the next interval and repeat until the desired intervals are set.
  5. Click on the feedback field for an interval to open an editor in which you enter feedback text for the interval.


Introduction Template

The following information provides users of the Performance Assessment Workbench (PAW) a template to formulate an introduction to the test and instructions on how to use the test player. Remember that the following test introduction is an example and is intended to be modified as required for each specific test. It is important to note that some of the functions or item formats described here do not appear in all types of test. In addition, this example introduction contains placeholders <> that are intended for authors to replace with their own relevant information.

Welcome to the <Test name> test!

Please be sure to read the following information completely before starting the test. The test time begins once you start the test at the end of this introduction.

ã This test contains proprietary software components and information of <customer> and SAP AG.

You can use the following options and functions to navigate through <test type/test name>:

The start button becomes active once you reach the last page of this introduction. When you click on this button, you simultaneously start <Test> and end the introduction. The test time begins running as soon as <Test> appears on your screen. You can display the introduction anytime while taking the test by clicking on the following icon in the test structure:

With this button you can exit the introduction without starting the test. When you access the test again, the introduction reappears.

With this button you can exit <Test> anytime after starting the test. All of your previous entries up to this point are saved. Please note that you cannot restart <Test> once you click this button.

You use the arrow buttons to jump to the previous or next item in the test. Your entry is automatically saved as soon as you click on one of the arrow buttons.

The test structure shows you the contents and organization of <Test>. You can click on the respective item icons to jump directly to an item. In addition, the test structure shows you which items from <Sub-test/Performance unit> you have already answered.

The answered items are shown with a yellow checkmark:

The items you looked at but did not answer are marked with a symbol with an eye:

The items you marked with the Mark item function, to review or correct later, are shown with a blue corner and a yellow checkmark.

In addition to the navigation functions, you can also use the following options and functions in the test player:

As described above, you select this option to mark an item for later review or correction before the test time runs out. Items you mark are represented in the test structure with the following icon:

This permanent time display informs you how much time is left to complete the test.

This media icon appears if you open an item that contains a screen shot, a graphic, or similar object. The media element appears automatically when you open the item containing it. However, if you close the window in which the media element appears, you can use this icon to open the window again.

General Information about <Test> Item Formats

<Test> contains <Number> items to be completed in <Total test time>. The following item formats are used in <Test>:

Multiple Choice

Multiple choice (MC) items can have one or more correct answers. You answer MC items by clicking on the button(s) next to the correct answer(s). Make sure you read the instructions for each item to determine whether more than one answer applies. In most cases, MC items with only one correct answer have radio buttons that you use to select an answer. That means if you have selected an answer and then select a different answer, the first answer alternative button is no longer marked. In this case, it is not possible for you to select more than one answer. This is not the case for MC items with more than one correct answer.

Multiple Choice Explicit

Multiple choice explicit (MC explicit) items have more than one correct answer. They differ from the MC item format in the number of selection buttons available for each answer alternative. MC explicit format is also known as true/false format. Each MC explicit item has two buttons for each answer alternative, so you must explicitly specify for each answer whether it is correct or incorrect.

Fill-In

For fill-in items, type your answer in the appropriate text field.

Rating:

<If used with test item formats: In contrast to item formats MC and MC explicit, rating items have no correct or incorrect answers.>

Rating items are scales, which you use to articulate your opinions, judgements, values, requirements or similar. You complete rating items by clicking on the scale rating that most closely represents your opinion, judgement, evaluation, or similar.

Test Feedback

Once <Test> is ended, either automatically or manually with the end test button, you receive feedback <about your test results>. You can click on an individual item in the test structure to see feedback specific to the item. You can return to the <Test results> by clicking the <Test icon > in the test structure.

Adding Remarks

Once <Test> is ended, either automatically or manually with the end test button, you can add your remarks to the individual items in the test. To do this, first click on the item icon in the test structure. Next click on the Remarks button and then type your text in the entry field that appears. Save your entries by clicking OK or cancel your entry by clicking Cancel.

General Information

Tests only:

Your entry is automatically saved as soon as you click on one of the arrow buttons or another item in the test structure.

While <Test> is running, you can change items you have already answered.

Once you press the start button, you can no longer stop the test time.

<Test> is automatically ended when the test time runs out.

<Good luck!>

Click on the start button when you are ready to end the introduction and start <Test>.


Sub-test Attributes

The sub-test attributes specify the properties or characteristics the sub-test assumes during test execution.

Most of the sub-test attributes are the same as the test attributes. The following describes the additional attributes specific to sub-tests:

  • Mastery required

This attribute determines whether successful completion (individual test value ³ cut-off value) of the sub-test is required for the successful completion of the entire test. If you specify that passing is required, the test is marked as "not passed", if either the individual test value is smaller than the cut-off value for the entire test or the sub-test was not successfully completed.
  • Label

This is a four-character maximum short descriptor used in the results display.

For more information, refer to the detailed attribute descriptions in Test Attributes .


Feedback Template A

Feedback Template A (Structured)

$TEST_ | TEST: |  | $TEST_DESCRIPTION
 | Your score: | $TEST_SCORE | $TEST_POSTEDITING_REQUIRED_ Note: This feedback does not reflect your final results since some items require manual rating. _TEST_POSTEDITING_REQUIRED$ $TEST_USE_SCORING_TABLE_ Your performance in this test is rated as: $TEST_SCORINGTABLE_FEEDBACK. _TEST_USE_SCORING_TABLE$ $TEST_PASSED_OK_ Congratulations! You passed this test. _TEST_PASSED_OK$ $TEST_PASSED_BAD_ Sorry, you did not pass this test. You can find additional training materials at $TEST_URL. _TEST_PASSED_BAD$
 | Maximum score: | $TEST_MAXSCORE
 | Score in percent: | $TEST_PERCENTSCORE
$SUBTEST_ |  |  |
 | Sub-test: |  | $SUBTEST_DESCRIPTION
 | Your score: | $SUBTEST_SCORE | $SUBTEST_USE_SCORING_TABLE_ Your performance in this sub-test is rated as: $SUBTEST_SCORINGTABLE_FEEDBACK. _SUBTEST_USE_SCORING_TABLE$ $SUBTEST_PASSED_BAD_ Since this sub-test is an essential component of the test and you did not reach the required $SUBTEST_CUTOFFVALUE % of this sub-test, you did not pass the test. _SUBTEST_PASSED_BAD$ You can find additional training materials for this sub-test at $SUBTEST_URL.
 | Maximum score: | $SUBTEST_MAXSCORE
 | Score in percent: | $SUBTEST_PERCENTSCORE
$PU_ |  |  |
 | Performance Unit: |  | $PU_DESCRIPTION
 | Your score: | $PU_SCORE | $PU_USE_SCORING_TABLE_ Your performance in this performace unit is rated as: $PU_SCORINGTABLE_FEEDBACK. _PU_USE_SCORING_TABLE$ You can find additional training materials for this performance unit at $PU_URL.
 | Maximum score: | $PU_MAXSCORE
 | Score in percent: | $PU_PERCENTSCORE
_PU$ _SUBTEST$ _TEST$ |  |  |


Feedback Template B

Feedback Template B (List)

$TEST_ | TEST: |  | $TEST_DESCRIPTION
 | Your score: | $TEST_SCORE | $TEST_POSTEDITING_REQUIRED_ Note: This feedback does not reflect your final results since some items require manual rating. _TEST_POSTEDITING_REQUIRED$ $TEST_USE_SCORING_TABLE_ Your performance in this test is rated as: $TEST_SCORINGTABLE_FEEDBACK. _TEST_USE_SCORING_TABLE$ $TEST_PASSED_OK_ Congratulations! You passed this test. _TEST_PASSED_OK$ $TEST_PASSED_BAD_ Sorry, you did not pass this test. You can find additional training materials at $TEST_URL. _TEST_PASSED_BAD$
 | Maximum score: | $TEST_MAXSCORE
 | Score in percent: | $TEST_PERCENTSCORE
_TEST$ |  |  |
$SUBTEST_ | Sub-test: |  | $SUBTEST_DESCRIPTION
 | Your score: | $SUBTEST_SCORE | $SUBTEST_USE_SCORING_TABLE_ Your performance in this sub-test is rated as: $SUBTEST_SCORINGTABLE_FEEDBACK. _SUBTEST_USE_SCORING_TABLE$ $SUBTEST_PASSED_BAD_ Since this sub-test is an essential component of the test and you did not reach the required $SUBTEST_CUTOFFVALUE % of this sub-test, you did not pass the test. _SUBTEST_PASSED_BAD$ You can find additional training materials for this sub-test at $SUBTEST_URL.
 | Maximum score: | $SUBTEST_MAXSCORE
 | Score in percent: | $SUBTEST_PERCENTSCORE
_SUBTEST$ |  |  |
$PU_ | Performance Unit: |  | $PU_DESCRIPTION
 | Your score: | $PU_SCORE | $PU_USE_SCORING_TABLE_ Your performance in this performace unit is rated as: $PU_SCORINGTABLE_FEEDBACK. _PU_USE_SCORING_TABLE$ You can find additional training materials for this performance unit at $PU_URL.
 | Maximum score: | $PU_MAXSCORE
 | Score in percent: | $PU_PERCENTSCORE
_PU$ |  |  |


Fehler vermeiden

PAW Author

Bevor Sie einen Test verwenden können, der mit dem PAW Author erstellt wurde:

  1. Stellen Sie sicher, daß Sie einen Test mit mindestens einer Testform erstellt haben
  2. Stellen Sie sicher, daß Sie eine Testform ausgewählt haben.
  3. Setzen Sie alle Info-Objekte, die zum Test gehören (Medias, Abbildungen, Items, Einführung, Feedback, Performance Units, Performance Unit Container, Test), auf 'freigegeben' (released)
  4. Wenn Sie einen neuen Performance Unit Container anlegen, setzen Sie das Attribut 'Usage Type' auf 'Level1'!

PAW Manager: Zertifizierungs-Veranstaltung pflegen

Voraussetzungen damit ein Test für eine Zertifizierungs-Veranstaltung von der Proctor-Software aus dem Knowledge Warehouse heruntergeladen werden kann:

  1. Es muss ein Test-Typ definiert sein (SL31)
  2. Der Test-Katalog muss gepflegt sein (SL32)
  3. Die Einstellungen für das Release Control müssen vorgenommen sein: Für den Test-Typ (SL31)
    sowie für Veranstaltung und Person (SL96)
  4. Mit der SL66 (Download booking data) und SL67 (Download Test Catalog) können Sie die gewählten Einstellungen überprüfen. Nur wenn Buchungsdaten und Test-Daten angezeigt werden, kann die Proctor-Software einen Test aus dem KW herunterladen.


Glossary

Finding terms

To search for a glossary term, enter the term in the text field of the Index tab.

To search for a term in the full text, enter the term in the text field of the Search tab, and press Enter or the pushbutton List Topics.

Displaying terms

To display a term highlighted in the list box, double-click on it or press the pushbutton Display.


HTML Help Viewer Help

Use

With the HTML Help Viewer you can view the SAP Library in file format Compiled HTML (*.CHM).

Features

The HTML Help Viewer comprises:

  • A topic window for displaying the current help document.
  • A navigation window for displaying the current table of contents, the keyword search and the full-text search. (The navigation window can be hidden or shown: Hide/Show.)
  • A toolbar containing navigation and print functions, etc.

If you go to other help files while navigating through the online help, other HTML Help Viewer instances are opened. These instances can be closed individually. If you close the initial instance, all viewer instances are closed.

To go to the first page of the SAP Library, choose Home. You have access to the entire documentation from there: SAP Library, Glossary, Implementation Guide (IMG) and Release Notes.

Finding Help Documents

To find help documents:

  • Navigate in the table of contents tree structure (Contents tab)
    • To expand or collapse all branches of the table of contents tree, click the right-hand mouse key in the navigation window and choose Expand all or Collapse all in the context menu.
    • To move backwards and forwards in a navigation path, choose Back or Forward in the toolbar. This applies to the navigation path within an HTML Help Viewer instance.
    • To display only the entries for particular info classes in the table of contents, click the right-hand mouse key in the navigation window and choose Customize... in the context menu. Specify in the following dialog which info classes are to appear in the table of contents.
  • Perform a keyword search (Index tab)

The complete index is displayed if you are in the SAP Library; the index of the current help file is displayed if you are in another help file. No Index tab is displayed if no index exists for the current help file.

Loading the complete SAP Library index (approx. 8 MB) can take several minutes, depending on the configuration of your PC.

Select an entry in the list of keywords and choose Display.
If the selected keyword is assigned to only one help document, that document is displayed. Otherwise a list of documents found is offered for selection.

As Release Notes and IMG have no index, only full-text search is possible for them.
  • Perform a full-text search (Search tab)

Enter a search request and choose List Topics.
All documents are searched if you are in the SAP Library; only the documents in the current help file are searched if you are in another help file. The documents found are listed for selection.

To make complex searches:
    • Use the place holder ? for single characters and * for character strings in search terms.
    • Match similar words
    • Combine search terms with the operators:
      • AND (all terms appear in the document)
      • OR (at least one of the terms appears in the document)
      • NOT (the term does not appear in the document)
      • NEAR (the terms occur close together, such as in the same sentence in the document)
    • Restrict the search to document titles (Search titles only)
    • Restrict the search to the set of documents found in the previous search (Search previous results)

A global search in the SAP Library, in particular full-text search, requires a large amount of main memory. Global full-text search should only be used if the help files are on the local hard disk.

Finding the Current Help Document in the Table of Contents

If you found the current topic by search or hyperlink, the position of the topic in the table of contents is not displayed. To find the topic in the table of contents, choose Locate in the HTML Help Viewer toolbar.

If you found the current topic by a global (full-text or keyword) search in the SAP Library, the help file containing the topic must first be found. Click on the Locate icon [image] in the topic title. Then choose Locate in the toolbar to expand the help file table of contents and find the position of the topic.

Printing Help Documents

You can print individual help documents or several documents at the same time:

  • To print the current topic, choose Print with the option Print the selected topic.
  • To print the current topic and all its subtopics, choose Print with the option Print the selected heading and all subtopics.


SAP Library
Basis Release 5.0A, 2000
SAP Library

Glossary

Getting Started

© Copyright 2000 SAP AG
All rights reserved.


Broken link

Error: Broken link

You tried to follow a link that points to an unknown target document.